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Introduction 


Executive  Summary 

The  Naval  Systems  Engineering  Guide  is  provided  to  help  ensure  the  systems  we  develop  for  the  fleet  are 
affordable,  operationally  effective  and  suitable,  and  can  be  a  timely  solution  to  satisfy  user  needs  at  an 
acceptable  level  of  risk.  This  Guide  defines  the  systems  engineering  (SE)  requirements  and  tasks;  their 
implementation  and  products;  and  explains  the  tools  and  techniques  used  throughout  a  product  life  cycle.  This 
Guide  satisfies  the  DoD  requirement  for  having  a  documented  SE  process,  and  emphasizes  the  relationship 
between  the  technical  management  process  and  the  SE  process.  It  documents  a  common  Naval  Systems 
Engineering  Process  that  has  been  accepted  by  the  Naval  Virtual  Systems  Command. 

The  purpose  of  this  Guide  is  to  characterize  the  contents  of  the  SE  discipline,  to  promote  a  consistent  and 
common  view  of  SE  across  the  Navy,  to  clarify  the  boundary  of  SE  with  respect  to  other  disciplines,  and  to 
provide  a  foundation  for  curriculum  development  and  SE  certification.  This  Guide  consists  of  information  and 
33  required  or  normative  processes.  This  Guide  describes  a  rigorous  process  to  assist  the  systems  engineer  in 
defining,  performing,  managing,  and  evaluating  SE  efforts  in  Naval  acquisition  and  technology  development 
programs.  The  intended  audience  is  the  new  systems  engineer,  an  engineer  in  another  discipline  that  needs  to 
perform  some  SE  functions,  or  a  more-experienced  systems  engineer  who  needs  a  convenient  reference.  The 
hyper-linking  to  the  imbedded  reference  material  makes  it  very  convenient  using  the  electronic  version  of  this 
Guide.  The  intent  is  to  provide  enough  information  for  the  user  to  determine  whether  a  given  process  activity  is 
appropriate  in  supporting  the  objective(s)  of  the  program  or  project  they  support,  and  how  to  go  about 
implementing  the  process  activity. 

The  framework  for  this  Guide  is  an  industry  standard,  ANSI/EIA-632,  Processes  for  Engineering  a  System. 

The  standard  was  developed  to  replace  the  SE  military  standard,  MIL-STD-499  as  part  of  the  1994  DoD 
Acquisition  Reform  initiative  prescribing  the  use  of  “performance-based”  acquisition  specifications  and  the 
substitution  of  the  standards  and  practices  used  in  the  commercial  marketplace  for  military  specifications  and 
standards.  The  Naval  Systems  Engineering  Steering  Group  (SESG),  comprised  of  members  from  NAVAIR, 
NAVSEA,  MARCOR,  and  SPAWAR,  provided  the  common  and  unique  SE  requirements  and  implementation 
approach  for  the  various  Naval  development  and  acquisition  programs.  Periodic  updates  are  planned  to 
implement  continuous  process  improvement,  based  upon  feedback  from  programs  /  contractors,  by  the  Naval 
SESG  which  maintains  this  Guide. 

It  is  expected  that  programs  would  adopt  the  process  in  this  Guide  and  tailor  the  specific  requirements  to  fit 
their  program  based  upon  where  the  program  is  in  terms  of  life  cycle,  technology  risks,  and  funding  levels. 
Though  there  is  an  attempt  made  to  show  how  products  are  affected  by  what  SE  process,  and  the  impact  to  the 
product  as  the  product  move  through  the  acquisition  phases,  the  emphasis  is  on  specifying  the  requirements  for 
the  processes  rather  than  phases.  Since  selection  of  an  acquisition  phase  is  dependent  on  the  particular 
application,  and  to  some  extent  organizational  structure,  specifying  temporal  flow  is  currently  outside  the  scope 
of  this  Guide. 

Background 

In  June  1994,  a  working  group  of  industry  associations,  the  International  Council  on  Systems  Engineering 
(INCOSE),  and  the  Department  of  Defense  developed  an  interim  standard  for  the  engineering  of  systems.  This 
effort  was  led  by  the  G-47  Committee  on  Systems  Engineering  of  the  Electronic  Industries  Alliance  (EIA).  The 
EIA/IS  632  was  intended  to  provide  a  standard  for  use  by  commercial  enterprises,  as  well  as  government 
agencies  and  their  development  contractors. 

In  April  1995,  a  formal  working  group  was  established  under  Project  PN-3537  and  with  EIA  and  INCOSE 
sponsorship  to  generate  and  release  this  full  Standard.  The  joint  working  group  decided  that  it  would  best  serve 
U.S.  industry  to  develop  a  “top-tier”  standard  applicable  across  all  industry  sectors  and  technology  domains.  As 
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a  result,  the  contents  of  this  Guide  are  an  abstraction  of  the  essential  features  of  the  engineering  practices 
described  in  the  interim  version  of  this  Standard. 

In  July  1999,  the  Naval  Air  Systems  Command  Systems  Engineering  Process  Working  Group  (NAVAIR 
SEPWG)  was  established  to  develop  a  NAVAIR  guide  to  EIA-632.  In  summer  2003,  the  Naval  Systems 
Engineering  Steering  Group  (SESG),  comprised  of  members  from  NAVAIR,  NAVSEA,  MARCOR,  and 
SPAWAR,  was  established  to  provide  the  common  and  unique  SE  requirements  and  the  implementation 
approach  for  the  various  Naval  development  and  acquisition  programs  in  this  Guide. 

This  Guide  is  consistent  with  ISO  9000  in  that  it  provides  processes  that  can  be  adopted  by  enterprises  for 
engineering  systems.  Appendix  A  is  normative.  Appendices  B  through  J  are  informative.  Appendices  beyond 
F  were  added  specifically  to  address  Naval  acquisition  and  development  resources. 

Compliance 

Use  of  this  of  this  Guide  is  provided  as  a  resource  for  documenting  the  Naval  information  process  and  is 
compliant  with  DoD  directives  on  having  a  documented  SE  process.  The  processes  of  this  Guide  should  be 
referenced  in  the  Systems  Engineering  Plan  (SEP)  as  required  by  DoD  directives. 

Users 

This  document  is  the  property  of  the  US  Navy  and  is  being  made  available  to  Naval  personnel.  Since  it 
contains  EIA-632  copyright  material,  arrangements  have  been  made  that  enable  customers,  contractor  personnel, 
partners,  and  non-governmental  team  members  are  able  to  obtain  and  use  this  document  within  their 
respective  organizations. 


This  Guide  assumes  that  the  reader  has  a  basic  understanding  of  systems  engineering  as  addressed  in  the 
DAWIA  Systems  201  and  SYS  301  courses,  and/or  other  graduate  level  SE  courses.  It  builds  on  those 
fundamentals,  and  concentrates  on  the  systems  engineering  process  and  the  relationship  and  interdependence  of 
these  process  steps.  This  Guide  attempts  to  use  the  most  common  terminology  of  the  SE  community  in  order  to 
facilitate  better  communication. 
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Document  Organization 

The  Guide  is  organized  as  follows: 


Section  1 

Scope 

states  the  purpose  of  this  Guide  and  defines  the  particular  processes 
to  which  it  is  intended  to  apply. 

Section  2 

Normative 

references 

lists  other  standards  that  are  so  referred  to  in  the  text  as  to  make 
them  indispensable  in  applying  this  Guide. 

Section  3 

Definitions  and 
acronyms 

defines  special  use  terms  and  acronyms. 

Section  4 

Processes 

contains  the  requirements  for  the  processes  that  are  central  to 
engineering  a  system.  Representative  tasks  associated  with  each 
process  are  defined. 

Section  5 

Application  context 

describes  the  context  in  which  the  processes  of  this  Guide  are 
applied. 

Section  6 

Application  key 
concepts 

describes  key  concepts  related  to  applying  the  processes  of  Section 

4  to  generate  and  integrate  the  layers  of  end  products  and  enabling 
products  needed  for  engineering  a  system. 

Appendix  A 

Glossary 

gives  definitions  for  words  that  are  used  in  a  specific  technical  way 
in  the  body  of  the  Guide.  Only  those  terms  for  which  the  normal 
dictionary  definition  does  not  suffice  are  included. 

Appendix  B 

Enterprise-based 

Life  Cycle 

describes  the  management-life-cycle  phases  in  which  a  system,  or 
portion  thereof,  is  incrementally  engineered. 

Appendix  C 

Process  Task 
Outcomes 

provides  expected  outcomes  for  the  representative  tasks  identified 
in  Section  4. 

Appendix  D 

Planning 

Documents 

lists  typical  source,  technical,  and  other  documents  related  to 
engineering  a  system  and  their  contents. 

Appendix  E 

System  Technical 
Reviews 

describes  the  necessary  technical  reviews  for  assessing  progress 
against  technical  plans  and  requirements,  and  for  assessing  planned 
tasks. 

Appendix  F 

Process 

Relationships 

defines  different  types  of  requirements  and  the  relationship  between 
these  types  and  the  logical  and  physical  solution  representations. 

Appendix  G 

Engineering 

Specialty 

References 

collection  of  engineering  specialty  references  listed  by  technical 
discipline. 

Appendix  H 

Naval  Process 

Flow  Diagrams 

collection  of  process  flow  diagrams  summarizing  the  33  Sub¬ 
processes 

Appendix  I 

Naval  Acronyms 

wording  for  acronyms 

Appendix  J 

Naval  References 

reference  listing  with  hyperlinks  to  electronic  versions  of 
documentation 
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1  Scope 


1.1  Purpose 

The  purpose  of  this  Guide  is  to  characterize  the  contents  of  the  SE  discipline,  to  promote  a  consistent  and 
common  view  of  SE  across  the  Navy,  to  clarify  the  boundary  of  SE  with  respect  to  other  disciplines,  and  to 
provide  a  foundation  for  curriculum  development  and  SE  certification. 

1.2  Applicability 

This  Guide  defines  processes  for  engineering  a  system.  These  have  been  organized  into  five  groups  as  shown  in 
Figure  1.1.  The  process  covered  in  the  legacy  MIL-STD-499B  is  noted  in  the  figure. 

Acquisition  and  Supply  (Subsection  4.1) 

♦  Supply  Process 

♦  Acquisition  Process 

Technical  Management  (Subsection  4.2) 

♦  Planning  Process 

♦  Assessment  Process 


♦  Systems  Analysis  Process 

♦  Requirements  Validation  Process 

♦  System  Verification  Process 

♦  End  Products  Validation  Process 

Figure  1.1  -  Fundamental  processes  for  engineering  a  system  with  MIL-STD-499B  comparison 

The  applicability  of  EAI  632  and  this  Guide  with  respect  to  enterprises  and  projects  is  shown  in  Figure  1.2. 

Industry  Naval  Enterprise  Program/Project 


establishes 

implements 

EIA  632 
Standard 

-► 

Policies  & 
Procedures 

-► 

Adopted 

Process 

Requirements 

Figure  1.2  -  Application  of  this  Guide 

The  EAI  632  standard  specifies  accepted  practices  used  for  engineering  systems  but  does  not  specify  the  details 
of  “how  to”  implement  process  requirements.  Nor  does  it  specify  the  methods  or  tools  a  developer  would  use  to 
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implement  the  process  requirements.  It  is  intended  that  the  developer  select  or  define  methods  and  tools  that  are 
applicable  to  the  development,  and  that  are  consistent  with  enterprise  policies  and  procedures. 

In  this  Guide,  the  Naval  policies  and  procedures  were  added  to  describe  “how  to”  with  respect  to  Naval 
programs.  The  intent  is  to  provide  the  Systems  Engineer  with  insight  into  how  Naval  Systems  Engineering 
processes  fit  into  the  overall  EIA-632  systems  engineering  framework.  Additionally,  whenever  possible, 
information  is  provided  regarding  the  inputs,  outputs,  entry  criteria,  exit  criteria,  references,  agents,  tools  and 
methods  that  Naval  engineering  may  use  to  accomplish  each  sub-process.  This  Guide  also  specifies  name, 
format,  content,  structure,  or  medium  for  documentation  for  NAVAIR  programs.  Since  selection  of  an 
acquisition  phase  is  dependent  on  the  particular  application,  and  to  some  extent  organizational  structure, 
specifying  temporal  flow  is  currently  outside  the  scope  of  this  Guide. 

It  is  expected  that  programs  would  adopt  the  process  in  this  Guide  and  tailor  the  specific  requirements  to  fit 
their  program  based  upon  where  the  program  is  in  terms  of  life  cycle,  technology  risks,  and  funding  levels.  The 
emphasis  is  on  specifying  the  requirements  for  the  processes  rather  than  phases. 


2  Normative  references 


The  normative  references  of  this  Guide  are  the  33  processes  constituting  Section  4  of  this  document.  These  are 
the  accepted  practices  used  for  engineering  systems  in  DoD  acquisition  programs.  These  normative  processes 
were  the  processes  adapted  from  EIA  632.  The  systems  engineering  process  timeline  as  it  applies  to  the  DoD 
acquisition  life  cycle  is  provided  in  Figure  3  a.  Additional  applicable  DoD  references  have  been  cited  in  each 
sub-process  for  the  execution  of  the  specific  process. 


3  Definitions  and  acronyms 

3.1  Key  terms 

Definitions  for  special  use  of  terms  are  contained  in  Appendix  A,  Glossary. 

3.2  Acronyms 

Acronyms  used  in  this  Guide  are  contained  in  Appendix  I,  Acronyms. 

3.3  Terminology 

The  word  shall  identifies  mandatory  provisions  of  this  Guide.  The  word  should  identifies  recommended 
provisions  of  this  Guide.  The  word  may  identifies  permissive  provisions  of  this  Guide. 
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Figure  3a  -  Systems  engineering  process  timeline 
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4  Processes 


This  section  provides  sub-processes  for  processes  used  in  engineering  a  system  and  is  applicable  to  any  product 
development  regardless  of  its  place  in  the  hierarchy  of  the  system  structure  (see  Section  6)  or  the  enterprise- 
based  life-cycle  phase  (see  Appendix  B).  The  processes  are  applicable  to  the  engineering  or  reengineering  of 
the  end  products  that  make  up  a  system,  as  well  as  the  development  of  enabling  products  required  to  provide 
life-cycle  support  to  system  end  products.  Figure  4a  shows  the  relationships  between  the  processes  of  this 
Guide. 


Acquisition 

Request 


System 

Products 


Figure  4a  -  Relationship  of  processes  for  engineering  a  system 
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NOTES 

1  The  application  of  the  processes  shown  in  Figure  4a  is  discussed  in  Section  6.  Appropriate  processes  of 
Figure  4a  are  applied  recursively  and  iteratively  to  define  the  system  products  of  the  system  hierarchy  from 
the  top  down,  and  then,  to  implement  and  transition  the  system  products,  from  the  bottom  up  to  the  user  or 
customer. 

2  Although  the  Sub-processes  in  this  Guide  are  presented  sequentially,  in  practice  many  associated  tasks 

are  concurrent  and  highly  iterative,  and  have  interactive  dependencies  that  lead  to  alteration  of  previously 
established  technical  requirements. _ 


This  Guide  specifies  the  33  Sub-processes  as  shown  in  Figure  4b. 


Supply  Sub-processes 

1  -  Product  Supply 

Acquisition 

Sub-processes 

2  -  Product  Acquisition 

3  -  Supplier  Performance 

Planning  Sub-processes 

4  -  Process  Implementation  Strategy 

5  -  Technical  Effort  Definition 

6  -  Schedule  and  Organization 

7  -  Technical  Plans 

8  -  Work  Directives 

Assessment 

Sub-processes 

9  -  Progress  Against  Plans  and  Schedules 

10  -  Progress  Against  Requirements 

1 1  -  Technical  Reviews 

CONTROL 

Sub-processes 

12  -  Outcomes  Management 

1 3  -  Information  Dissemination 


Requirements  Definition 
Sub-processes 

14  -  Acquirer  Requirements 

15  -  Other  Stakeholder  Requirements 

16  -  System  Technical  Requirements 

Solution  Definition 
Sub-processes 

17  -  Logical  Solution  Representations 

1 8  -  Physical  Solution  Representations 

19  -  Specified  Requirements 

Implementation 

Sub-processes 

20  -  Implementation 

Transition  to  Use 
Sub-processes 

21  -  Transition  to  Use 


Systems  Analysis 
Sub-processes 

22  -  Effectiveness  Analysis 

23  -  Trade-off  Analysis 

24  -  Risk  Analysis 

Requirements  Validation 
Sub-processes 

25  -  Requirements  Statements  Validation 

26  -  Acquirer  Requirements  Validation 

27  -  Other  Stakeholder  Requirements 
Validation 

28  -  System  Technical  Requirements 
Validation 

29  -  Logical  Solution  Representations 
Validation 

System  Verification 
Sub-processes 

30  -  Design  Solution  Verification 

3 1  -  End  Product  Verification 

32  -  Enabling  Products  Readiness 

End  Products  Validation 
Sub-processes 

33  -  End  Products  Validation 


Figure  4b  -  Sub-processes  for  engineering  a  system 

The  developer  should:  (1)  decide  which  of  the  processes  in  Figure  4a  apply  to  their  enterprise;  (2)  decide  which 
sub-processes  from  this  Guide  apply  for  the  processes  selected;  (3)  establish  appropriate  policies  and  procedures 
that  govern  project  implementation;  (4)  define  appropriate  tasks  for  each  of  the  selected  sub-processes;  and  (5) 
establish  methods  and  tools  to  support  task  implementation.  Representative  tasks,  along  with  their  expected 
outcomes,  are  provided  in  Appendix  C  for  each  sub-process  of  this  Guide. 


NOTES 

1  The  developer  can  be  an  enterprise,  a  group  of  enterprises,  an  organization  or  a  project. 

2  A  developer  can  be  either  an  acquirer  or  a  supplier  of  systems,  subsystems,  or  end  products. 

A  developer  can  act  in  both  roles  (acquirer  and  supplier)  simultaneously  on  the  same  project,  e.g.,  supplying  an  end 
product  to  another  organization,  while  acquiring  subsystems  from  a  third  organization. 


For  a  system  that  contains  product  elements  for  which  lower-tier  development  standards  exist,  or  where 
standards  or  guides  exist  for  safety,  security,  or  other  system  aspects,  these  should  be  used  in  conjunction  with 
this  Guide  -  for  example:  (1)  IEEE/EIA  12207  for  a  system  that  contains  software,  or  for  a  stand-alone  software 
product;  and  (2)  ANSI/EIA-649  for  configuration  management. 
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4.1  Acquisition  and  Supply 


The  Acquisition  and  Supply  Processes  are  used  by  a  developer  to  arrive  at  an  agreement  with  another  party  to 
accomplish  specific  work  and  to  deliver  required  products,  or  with  another  party  or  parties  to  have  work  done  to 
obtain  desired  products.  The  parties  can  either  be  inside  the  developer’s  own  enterprise  (another  project, 
functional  organization,  or  project  team),  or  can  be  in  a  different  enterprise.  The  Acquisition  and  Supply 
Processes  can  be  initiated  as  a  result  of  a  project  go-ahead  or  approval  decision,  or  by  the  receipt  of  an 
acquisition  request,  offer  or  directive.  A  project  go-ahead  can  be  given  within  an  enterprise  as  a  result  of  a 
market-needs  analysis,  technology  breakthrough,  a  perceived  market  opportunity,  a  customer  requirement,  an 
internal  project  directive,  or  similar  stimulus. 


NOTE  -  Although  a  project  or  development  effort  can  be  initiated  by  casual  means,  an  agreement  is 
nevertheless  useful  to  ensure  that  all  parties  involved  understand  the  purpose,  goals,  and  expectations  of  the 
work. 


An  agreement  can  be  between  enterprises  and  between  organizational  elements  within  an  enterprise,  to  include 
between  projects,  between  projects  and  functional  units,  and  between  units  within  a  project.  The  agreement 
within  an  enterprise  can  take  the  form  of  a  work  directive,  work  package,  work  authorization,  or  project 
memorandum  of  agreement.  Agreements  between  enterprises  can  take  the  form  of  a  formal  contract  for  the 
delivery  of  a  product,  or  a  memorandum  of  agreement  that  establishes  the  working  relationship  between  two  or 
more  enterprises  on  a  common  project. 

Regardless  of  the  form  or  purpose  of  the  agreement,  certain  information  should  be  included,  for  example: 

a)  Work  to  be  performed; 

b)  Cost  and  schedule  constraints; 

c)  Concept  of  operations; 

d)  Requirements  to  be  satisfied,  including  known  functional,  performance,  and  interface  requirements, 
attributes,  and  characteristics; 

e)  Product  and  data  to  be  delivered; 

f)  Information  pertaining  to  the  cost,  schedule,  planning,  delivery  information,  training  and  user  manual, 
product  structure,  packaging  and  handling  instructions,  or  installation  instructions; 

g)  Appropriate  technical  plans; 

h)  Applicable  financial  structure,  management  and  authority  provisions; 

i)  Exit  criteria  for  relevant  enterprise-based  life-cycle  phases; 

j)  Identification  of  applicable  engineering  life-cycle  phases; 

k)  Required  technical  reviews. 


NOTE  -  a  developer  can  be  developing  a  product  with  out  any  contractual  relationship  to  the  user  or  customer  (e.g., 
commercial  product  development).  However,  much  of  the  information  above  must  be  available  to  the  developing 
organization  in  order  to  proceed. 


The  role  of  the  developer  with  respect  to  the  two  processes  of  Acquisition  and  Supply  is  shown  in  Figure  4.1. 
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Acquisition 

Requirements 


Agreement 


Figure  4.1  -  Acquisition  and  Supply  Processes 


NOTES 

1  The  acquirer  can  be  either  one  of  the  following: 

a.  Internal  to  enterprise  -  for  example,  another  project,  marketing  organization,  parent  project  of  a  product 
team,  the  project  team  itself,  executive  manager,  supervisor. 

b.  External  to  enterprise  -  for  example,  procurement  agency,  prime  contractor,  another  developer,  buyer, 
customer,  end  user,  owner,  purchaser. 

2  The  supplier  can  be  either  one  of  the  following: 

a.  Internal  to  enterprise  -  for  example,  another  project,  functional  organization,  product  team. 

b.  External  to  enterprise  -  for  example,  another  developer,  prime  contractor,  producer,  seller,  subcontractor, 
vendor. 

The  sub-processes  of  this  Guide  apply  to  the  developer  in  its  role  as  acquirer,  supplier,  or  both. 


4.1.1  Supply  Process 

This  process  is  used  by  the  developer  when  acting  as  a  supplier  to  establish  and  satisfy  an  agreement  with  the 
acquirer. 


Sub-process  1  -  Product  Supply 

For  a  system,  or  portion  thereof,  supplied  to  an  acquirer,  the  developer  (when  acting  as  the 
supplier)  shall  establish  and  satisfy  an  agreement  with  the  acquirer. 


The  supplier  is  typically  thought  of  as  a  Prime  Contractor,  but  may  be  a  team  within  DoN  or  another 
government  activity. 

Preceding  Process 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  8:  Work  Directives 

Inputs 
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Sub-process  1 


•  Acquisition  Strategy  (SP  2) 

•  Solicitation  (RFP,  SOW  or  SOO  with  Cost/Schedule  Requirements)  (SP  2) 

•  Acquirer  Offer  (SP  2) 

•  Requests  for  clarification  (SP  2) 

•  Request  for  Information  (RFI)  (SP  2) 

•  Acquirer  Signed  Agreement  (contract  or  program  directive)  (SP  2) 

NAVAIR  Specific: 

•  Team  Assignment  Agreement  (TAA)  (SP  8) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  (as  supplier)  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to 
consider  include  the  following: 


a)  Assess  the  acquisition  request,  offer,  or  directive  to  determine  the  capability  to  meet  the 
acquisition  document  requirements.  Supplier  develops  business  strategy  and  surveys  marketplace 
for  business  opportunities  (Commerce  Business  Daily  (CBD)/Federal  Business  Opportunities 
(FedBizOpps/FBO)  announcements/Sources  Sought,  etc.).  Supplier  obtains  Request  for 
Proposal/Quotation  and  allocates  resources  to  review  Request  for  Proposal/Quotation.  For  larger 
procurements  the  supplier  would  put  together  a  team  of  personnel  from  various  disciplines  such  as 
engineering,  financial,  logistics,  and  management.  For  some  efforts,  a  field  activity  may  be  used  -  in 
the  case  of  NAVAIR,  a  Naval  Air  Systems  Command,  Team  Assignment  Agreement  (TAA)  would  be 
used.  In  the  event  another  military  service  is  used,  a  MIPR  (Military  Interservice  Procurement 
Request)  would  be  used.  The  team  would  review  the  RFP,  determine  what  the  requirements  are,  and 
then  come  up  with  their  solution  to  meet  all  the  requirements  of  the  proposal.  Some  of  the  items  that 
may  be  included  in  their  proposal  would  include: 

•  executive  overview 

•  technical  approach 

•  systems  engineering 

•  producibility 

•  cost 

•  schedule 

•  performance 

•  specifications 

•  training 

•  program  management 

•  support  equipment  (common  and  peculiar) 

•  technology  risks 

•  human  systems  integration 

•  packaging  and  handling 

•  technical  data 

•  configuration  management  approach 

•  work  breakdown  structure 

•  site  activation 

•  industrial  facilities 

•  initial  spares  and  initial  repair  parts 

b)  Establish  a  satisfactory  agreement  within  legal,  regulatory,  enterprise,  and  project  bounds. 

Supplier  determines  if  the  capability  to  meet  the  acquisition  requirements  exists,  allocates  resources 
needed  to  prepare  the  proposal/quotation,  prepares  proposal/quotation,  submits  (or  presents  orally) 
proposal/quotation,  responds  to  proposal/quotation  clarification  questions  from  acquirer,  and  modifies 
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proposal  in  response  to  acquirer  requests.  The  established  agreement  would  also  delineate  any 
subcontracting  that  the  prime  contractor  may  enter  into  and  any  flowdown  requirements. 


c)  Record  the  established  agreement  in  the  form  appropriate  to  the  effort.  Supplier  and  acquirer 
negotiate  contract  terms.  Supplier  may  have  to  prepare  Best  and  Final  Offer. 

d)  Implement  the  processes  of  this  Guide,  as  applicable,  to  meet  the  requirements  of  the  agreement 

(contract  performance).  Supplier  and  acquirer  sign  contract. 

e)  Deliver  the  products  and  other  deliverables  as  specified  in  the  established  agreement.  Supplier 
performs  work  required  by  the  contract,  while  acquirer  monitors  supplier's  performance  and 
compliance  with  requirements.  Supplier  develops  and  documents  the  final  product  design.  Supplier 
manufactures  and  tests  product.  Supplier  develops  required  product  documentation  and  other  technical 
data  as  delineated  in  the  Supplier  Signed  Agreement. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Supplier  Proposal  (SP  2) 

•  Supplier  Signed  Agreement  (contract  or  program  directive)  (SP  2) 

•  End  Products  (SP  2,  3,  20,  31,  33) 

•  Enabling  Products  (SP  2,  3,  20,  31,  32,  33) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents.  (Products/Deliverables  meet  Agreement 
Requirements) 


Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Control  Process 

Sub-process  12:  Outcomes  Management 
Implementation  Process 

Sub-process  20:  Implementation 
System  Verification  Process 

Sub-process  31:  End  Product  Verification 
Sub-process  32:  Enabling  Products  Readiness 
End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Agents 

Contracts 

Systems  Engineering 

Logistics/R&M 

Business  Development 

Acquirer 

Manufacturing 

Technical  Writer 

Legal 

Security 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

Make  versus  Buy 

PRWeb 
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References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
MIL-STD-961 
MIL-HDBK-245 

SD-2  Buying  Commercial  and  Non-Developmental  Items:  A  Handbook 

SD-5  Market  Research 

OSD  Commercial  Item  Acquisition:  Considerations  and  Lessons  Learned,  26  June  2000 
Managing  Quality  and  Productivity  in  Aerospace  and  Defense,  November  1989 

NAVAIR  Specific: 

•  NAVAIRINST  5400.154 


Metrics  and  Measures 

•  Timeliness  of  the  supplier  against  the  completion  of  the  contract,  task  orders,  milestones,  delivery 
schedules,  administrative  requirements,  etc. 

•  Compliance  with  technical  performance  requirements. 

•  Effectiveness  in  forecasting,  managing,  and  controlling  contract  cost. 

•  Management  Responsiveness  -  Timeliness,  completeness,  and  quality  of  problem  identification,  corrective 
action  plans,  proposal  submittals  (especially  responses  to  change  orders,  engineering  change  proposals,  or 
other  undefmitized  contract  actions),  the  contractor’s  history  of  reasonable  and  cooperative  behavior, 
effective  business  relations,  and  customer  satisfaction. 

•  Subcontract  Management  -  timeliness  of  award  and  management  of  subcontracts,  including  whether  the 
contractor  met  small/small  disadvantaged  and  women-owned  business  participation  goals. 

•  Program  Management  and  Other  Management  -  Assess  the  extent  to  which  the  supplier  discharges  their 
responsibility  for  integration  and  coordination  of  all  activities  needed  to  execute  the  contract;  identifies  and 
applies  resources  required  to  meet  schedule  requirements;  assigns  responsibility  for  tasks/actions  required 
by  the  contract;  and  communicates  appropriate  information  to  affected  program  elements  in  a  timely 
manner.  Assess  the  supplier’s  risk  management  practices,  especially  the  ability  to  identify  risks  and 
formulate  and  implement  risk  mitigation  plans.  If  applicable,  identify  and  assess  any  other  areas  that  are 
unique  to  the  contract,  or  that  cannot  be  captured  elsewhere  under  the  Management  element. 

•  Number  and  severity  of  discrepancies  documented  during  product  verification. 

•  Number  and  severity  of  unresolved  discrepancies. 

•  Acceptance  of  test  results. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C,  Table  C.l.  The  outcomes 

associated  with  completing  this  sub-process  influence  the  Acquisition,  Planning,  and  Control  Processes  and  will 

flow  to  all  areas. 


4.1.2  Acquisition  Process 

The  developer  when  acting  as  an  acquirer  to  establish  an  agreement  with  a  supplier  and  to  manage  supplier 
performance  uses  this  process. 
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The  Acquisition  Process  includes  the  two  sub-processes  shown  in  Figure  4.1.2. 


Acquisition 

Process 

Requirements 


Sub-process  2  -  Product  Acquisition 


Sub-process  3  -  Supplier  Performance 


Figure  4.1.2  -  Acquisition  Process/Sub-processes 


Sub-process  2  -  Product  Acquisition 

For  a  system,  or  portion  thereof,  acquired  from  a  supplier,  the  developer  (when  acting  as  the 
acquirer)  shall  establish  an  agreement  with  that  supplier. 


The  supplier  is  typically  thought  of  as  a  Prime  Contractor,  but  may  be  a  team  within  DoN  or  another 
government  activity.  The  acquisition  may  be  competitive  or  sole  source.  There  are  different  procedures,  which 
must  be  followed  depending  on  whether  the  acquisition  is  competitive  or  sole  source. 

For  major  weapon  systems,  the  acquisition  process  initiates  within  the  service  or  field  commander-in-chiefs 
ongoing  mission  area  need  analysis  effort,  which  may  result  in  an  Initial  Capabilities  Document  (ICD)  - 
formerly  Mission  Need  Statement  (MNS).  By  certifying  a  mission  need,  the  ICD  may  result  in  a  Concept 
Decision  to  explore  material  solutions.  The  program  then  enters  the  Concept  Refinement  Phase,  during  which 
system  alternatives  are  explored.  The  next  phase  occurs  after  Milestone  A,  and  is  known  as  Technology 
Development  (formerly  Component  Advanced  Development  (CAD)).  The  preferred  system  concept  is  defined 
by  a  set  of  system  performance  requirements,  and  the  technology  is  demonstrated  to  show  that  any  significant 
technical  and  acquisition  risk  areas  identified  have  been  brought  under  sufficient  control  to  warrant  entering  the 
next  program  phase.  Program  Initiation  begins  at  Milestone  B,  which  is  the  beginning  of  the  System 
Development  and  Demonstration  (SDD)  (formerly  EMD)  Phase.  The  SDD  Phase  includes  the  System 
Integration  and  the  System  Demonstration  Work  Efforts,  which  are  separated  by  the  programmatic  Design 
Readiness  Review,  and  the  product  Critical  Design  Review  (CDR).  The  preliminary  design  and  detailed 
designs  are  completed  during  the  System  Integration  Work  Effort,  and  tests  are  performed  during  the  System 
Demonstration  Work  Effort. 

Following  the  Milestone  C,  the  system  enters  the  Production  and  Deployment  phase,  during  which  low-rate 
initial  production  and  full-rate  production  takes  place.  After  Initial  Operating  Capability  (IOC)  occurs,  the 
Operations  and  Support  phase  is  entered,  modifications  and  product  improvements  are  usually  implemented.  At 
the  end  of  the  system  service  life  it  is  disposed  of  in  accordance  with  applicable  classified  and  environmental 
laws,  instructions,  regulations,  and  directives.  Disposal  activities  also  include  recycling,  material  recovery, 
salvage  reuse,  and  disposal  of  by-products  from  development  and  production. 

At  the  conclusion  of  the  first  three  phases,  the  requirement  for  the  program  is  re-certified  by  the  Milestone 
Decision  Authority  (MDA)  before  additional  resources  are  authorized.  At  each  review,  the  decision  authority 
may  also  direct  a  tailored  program  to  omit  or  combine  specific  phases.  These  special  cases  are  normally  based 
on  the  decision  authority  being  convinced  that  the  technology  and  design  maturity  support  such  a  decision.  (For 
additional  information  see  the  Defense  Acquisition  University,  Systems  Engineering  Fundamentals,  Dec  2000, 
Defense  Acquisition  University  Press). 

Preceding  Process 
Supply  Process 

Sub-process  1 :  Product  Supply 
Acquisition  Process 

Sub-process  3 :  Supplier  Performance 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
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Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Sub-process  8:  Work  Directives 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Inputs  (“EXT”  indicates  it  is  external,  unspecified,  and  not  from  a  sub-process.) 

•  Supplier  Proposal  (SP  1) 

•  Supplier  Signed  Agreement  (contract  or  program  directive)  (SP  1) 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Supplier  Performance  Management  Plan  (SP  3) 

•  Work  Breakdown  Structure  (WBS)  (SP  5) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Test  and  Evaluation  Management  Plan  (TEMP)  (SP  7) 

•  Source  Selection  Plan  (SSP)  (SP  7) 

•  Team  Work  Plan  (TWP)  (SP  8) 

•  Statement  of  Objectives  (SOO)  (SP  8) 

•  Statement  of  Work  (SOW)  (SP  8) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Capability  Development  Document  (CDD)  or  Capability  Production  Document  (CPD)  -  formerly 
Operational  Requirements  Document  (ORD)  (SP  14) 

•  Specified  Requirements  (SP  19) 

•  Operational  Test  Readiness  Review  (OTRR)  certification  message  (SP  33) 

•  Cost,  Schedule,  and  Performance  constraints  (EXT) 

•  Acquisition  Strategy  (EXT) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  (as  acquirer)  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  Tasks  to 
consider  include  the  following: 

a)  Prepare  the  applicable  acquisition  request,  offer,  or  directive  to  obtain  supply  of  work  or 
delivery  of  desired  system  products. 

1)  The  contracting  process  begins  with  planning  efforts.  Planning  includes  development  of  a  Request 
for  Proposal  (RFP),  specifications  (Sub-process  19),  a  Statement  of  Work  (SOW)  or  Statement  of 
Objectives  (SOO),  a  Source  Selection  Plan  (SSP),  and  the  Contract  Data  Requirements  List 
(CDRL).  The  SOW  is  a  statement  of  the  work  to  be  done.  A  SOO  can  be  utilized  to  obtain  a  SOW 
or  equivalent  during  the  selection  process. 

2)  The  RFP  is  the  solicitation  for  proposals.  The  government  distributes  it  to  potential  contractors. 
The  RFP  delineates  the  need  and  what  the  offeror  must  do  to  be  considered  for  the  contract.  It 
establishes  the  basis  for  the  contract  that  will  be  put  in  place. 

3)  The  information  required  to  be  in  the  proposals  responding  to  the  solicitation  is  also  key  for  the 
systems  engineer.  The  engineering  team  decides  the  technical  and  technical  management  merits  of 
the  proposals.  The  directions  to  the  offerors  must  be  clearly  and  correctly  stated,  otherwise  the 
proposal  will  not  contain  the  information  needed  to  evaluate  the  offerors. 

4)  The  acquisition  package  contains  the  documents  that  will  be  provided  to  the  offerors  as  part  of  the 
RFP.  The  RFP  normally  includes: 
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•  Contract  Data  Requirements  List  (DD  Form  1423) 

•  Contract  Schedule,  Specification 

•  SOW  (Statement  of  Work)  or  SOO  (Statement  of  Objectives) 

•  Proposal  Requirements 

•  Contract  Security  Classification  (DD  Form  254) 

•  Supplier  Performance  Management  Plan  (optional  but  recommended) 

There  are  other  documents  that  are  part  of  the  Acquisition  Package,  which  are  kept  internal  to  the 
Government  and  must  remain  as  part  of  the  contract  file.  These  documents  typically  include: 

•  Procurement  Request 

•  Funding  Authorization  Document 

•  Procurement  Planning  Schedule 

•  Source  List 

•  Proposal  Evaluation  Plan 

A  description  of  the  various  types  of  acquisition  packages  and  their  content  may  be  found  at 

http  ://www.ntsc.na  vy  .mil/Resources/Library/Acq  guide/Acq  guide.htm 

Another  source  for  information  is  available  at  the  Navy’s  Acquisition  Reform  web  site 
http://www.acq-ref.navy.mil/index.html 

b)  Evaluate  supplier  response  to  acquisition  request,  offer,  or  directive.  The  process  begins 
with  the  development  of  a  Source  Selection  Plan  (SSP),  which  relates  the  organizational  and 
management  structure,  the  evaluation  factors,  and  the  method  of  evaluating  the  offerors’  responses. 

The  evaluation  factors  and  their  priority  are  transformed  into  information  provided  to  the  offerors  in 
sections  L  and  M  of  the  RFP.  The  offeror’s  proposals  are  then  evaluated  with  the  procedures 
delineated  in  the  SSP.  These  evaluations  establish  which  offerors  are  conforming,  guide  negotiations, 
and  are  the  major  factor  in  contractor  selection.  The  system  engineering  area  of  responsibility  includes 
support  of  SSP  (Source  Selection  Plan)  development  by  preparing  the  technical  and  technical 
management  parts  of  evaluation  factors;  organizing  technical  evaluation  teams;  and  developing 
methods  to  evaluate  the  offeror’s  proposals  (technical  and  technical  management). 

Source  selection  determines  which  offeror  will  be  the  contractor,  so  this  decision  will  have  profound 
impact  on  program  risk.  The  systems  engineer  should  approach  the  source  selection  with  great  care 
since,  unlike  many  planning  decisions  made  early  in  product  life  cycles,  the  decisions  made  relative  to 
source  selection  can  generally  not  be  easily  changed  once  the  process  begins.  Laws,  regulations, 
directives,  and  instructions  governing  the  fairness  of  the  process  require  that  changes  be  made  very 
carefully,  and  frequently  at  the  expense  of  considerable  time  and  effort  on  the  part  of  program 
management  and  contractor  personnel.  In  today’s  environment,  even  minor  mistakes  can  cause 
distortion  of  proper  selection.  Because  of  the  importance  of  this  process  NAVAIR  has  a  source 
selection  office  (AIR-4. 10E)  chartered  with  the  responsibility  to  ensure  the  source  selection  process  is 
properly  executed. 

c)  Make  offer  or  provide  directive  to  desired  supplier.  After  the  source  selection  is  completed,  an 
offer  is  made  or  directive  provided  to  the  selected  contractor(s). 

d)  Negotiate  agreement  to  establish  a  satisfactory  agreement  within  legal,  regulatory,  enterprise, 
and  project  bounds.  A  satisfactory  agreement  is  established  based  on  the  bounds  determined  by,  as 
appropriate: 

1)  applicable  legal,  regulatory,  policies,  procedures,  directives,  instructions  and  practices  that  will 
affect  negotiation  strategy; 

2)  the  type  of  agreement  to  be  negotiated; 

3)  negotiation  strategy; 
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4)  conditions  identified  from  the  plans  for  the  procurement  work  effort  that  could  affect  negotiations 
and  agreement  performance;  and 

5)  constraints  identified  from  the  plans  for  the  procurement  work  effort  that  could  affect  negotiations 
and  agreement  performance. 

e)  Record  the  established  agreement  in  the  form  appropriate  to  the  effort  (goes  to  Sub-process  12).  Upon 
completion  of  the  source  selection  process,  and  after  any  negotiations  are  finished,  a  contract  is  prepared 
and  sent  to  the  contractor(s)  for  signature.  After  the  contractor  signs,  the  contract  is  returned  to  the  PCO 
(Procurement  Contracting  Officer)  for  signature  on  behalf  of  the  government.  Once  the  contract  has 
been  signed  by  the  contractor  and  government,  its  terms  and  conditions  are  enforceable  by  law. 

f)  Accept  delivered  products.  Installed  or  delivered  system  products  must  be  validated  as  satisfying  user, 
customer,  or  assigned  requirements,  and  meeting  other  applicable  certification  or  acceptance  criteria.  A 
DD  Form  250  is  frequently  used  to  accept  deliveries  on  behalf  of  the  government. 

Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Cost,  schedule,  and  performance  constraints  (SP  5,  8) 

•  Acquisition  Strategy  (SP  1,  5,  6) 

•  Solicitation  (RFP,  SOW  or  SOO  with  Cost/Schedule  Requirements)  (SP  1,  3,  5) 

•  Acquirer  Offer  (SP  1) 

•  Request  for  Clarification  (SP  1) 

•  Request  for  Information  (RFI)  (SP  1) 

•  Acquirer  Signed  Agreement  (contract  or  program  directive)  (SP  1) 

•  ILS  Certification  (SP  21) 

•  Signed  DD  Form  250(s)  (SP  21) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents.  (Products/Deliverables  meet  Agreement 
Requirements) 

Next  Processes 
Supply  Process 

Sub-process  1 :  Product  Supply 
Acquisition  Process 

Sub-process  3 :  Supplier  Performance 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  6:  Schedule  and  Organization 
Sub-process  8:  Work  Directives 
Control  Process 

Sub-process  12:  Outcomes  Management 
Transition  to  Use  Process 

Sub-process  21:  Transition  to  Use 

Agents 
Contracts 
Source  Selection 
Legal 

Program  Manager  (PM) 

System  Engineering 

Logistics 

T&E 

Tools 
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Specifications 

PRWeb 

Proposal  Evaluation  Report 
Turbo  Streamliner 
Turbo  Specright! 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
MIL-STD-961D 

MIL-HDBK-245 

MIL-STD-499B 

SD-2  Buying  Commercial  and  Non-Developmental  Items:  A  Handbook 

SD-5  Market  Research 

Capability  Maturity  Model®Integration  (CMMI— \  2001:  Supplier  Agreement  Management  and 
Integrated  Supplier  Management  process  areas 

OSD  Commercial  Item  Acquisition:  Considerations  and  Lessons  Learned,  26  June  2000 
Managing  Quality  and  Productivity  in  Aerospace  and  Defense,  November  1989 

DP  Form  1423 

DP  Form  254 

DP  Form  250 


Metrics  and  Measures 

Metrics  are  measurements  collected  for  the  purpose  of  ascertaining  project  progress  and  overall  condition  by 
observing  the  change  of  the  measured  quantity  over  time.  Measurement,  evaluation  and  control  of  metrics  are 
normally  attained  through  a  system  of  periodic  reporting  that  must  be  planned,  established,  and  monitored  to 
assure  metrics  are  properly  measured,  evaluated,  and  the  resulting  data  disseminated. 

IPT  Participation,  Review  and  Concurrence  -  The  IPT  should  be  involved  from  program  initiation  and  during 
reviews  -  there  should  be  a  consensus  from  the  IPT  at  each  step  along  the  way. 

Technical  Reviews  -  typical  system-level  technical  reviews  (described  in  Sub-process  11)- 

•  Alternative  System  Review 

•  System  Requirements  Review 

•  System  Functional  Review 

•  Preliminary  Design  Review  (Includes  System  Software  Specification  Review) 

•  Critical  Design  Review 

•  Test  Readiness  Review 

•  Production  Readiness  Review 

•  System  Verification  Review 

•  Functional  Configuration  Audit 

•  Physical  Configuration  Audit/Review 

Product  Metrics  -  track  key  attributes  of  the  design  to  examine  progress  toward  meeting  customer  requirements. 
Product  metrics  reflect  three  basic  types  of  requirements: 

•  Operational  Performance 

•  Life-cycle  Suitability 
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•  Affordability 

Measures  of  Effectiveness  (MOEs):  Metrics  used  to  measure  results  achieved  in  overall  mission  and  execution 
of  tasks.  MOEs  are  a  prerequisite  to  the  performance  of  combat  measurement  (CJCSI  3170.01C). 

Measures  of  Performance  (MOPs)  -  measures  a  system’s  technical  performance  expressed  as  speed,  payload, 
range,  time  on  station,  frequency,  or  the  distinctly  quantifiable  performance  features.  Several  MOPs  may  be 
releated  to  the  achievement  of  a  particular  MOE. 

Technical  Performance  Measurements  (TPM)  -  derived  directly  from  MOPs  and  are  selected  as  being  critical 
from  a  periodic  review  and  control  perspective. 

Suitability  Metrics  -  tracking  metrics  relating  to  operational  suitability,  and  other  life  cycle  concerns  may  be 
appropriate  to  monitor  progress  toward  an  integrated  design.  Operational  suitability  is  the  degree  to  which  a 
system  can  be  placed  satisfactorily  in  field  use  considering  availability,  compatibility,  interoperability, 
transportability,  human  factors,  reliability,  maintainability,  documentation,  safety,  training,  manpower, 
supportability,  logistics,  usage  rates,  and  environmental  impacts. 

Product  Affordability  -  estimated  unit  production  cost  can  be  tracked  during  the  design  effort  in  a  manner 
similar  to  the  TPM  approach,  with  each  Cl  (Configuration  Item)  element  reporting  an  estimate  based  on  current 
design. 

Timing  -  product  metrics  are  tied  directly  to  the  design  process.  Planning  for  metric  identification,  reporting, 
and  analysis  is  started  with  initial  planning  in  the  concept  exploration  phase. 

Earned  Value  -  reporting  system  that  uses  cost-performance  metrics  to  track  the  cost  and  schedule  progress  of 
system  development  against  a  projected  baseline.  It’s  a  “big  picture  approach”  and  integrates  concerns  related 
to  performance,  cost  and  schedule. 

Process  Metrics  -  management  process  metrics  are  measurements  taken  to  track  the  process  of  developing, 
building,  and  introducing  the  system. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  influence  the  Supply,  Planning,  and  Control  Processes. 


Sub-process  3  -  Supplier  Performance 

The  developer  (when  acting  as  the  acquirer)  shall  manage  supplier  performance  (and  sub¬ 
suppliers)  to  ensure  that  the  technical  effort  to  be  accomplished  by  the  supplier  provides  end 
products  that  satisfy  the  assigned  requirements. 


The  focus  of  this  task  is  to  Manage  Supplier  Performance  by  monitoring  the  supplier  against  key  product  and 
process  metrics  that  can  include  periodic  reviews  (i.e.,  incoming  and  final  inspection,  facility  capability  audits, 
and  process  capability  studies).  Sub-process  3  is  invoked  whenever  subsystem  products  are  acquired  from 
suppliers  or  lower-tier  developers  outside  the  enterprise,  as  well  as  when  the  supplier  is  an  organizational  entity 
within  the  developer’s  own  enterprise. 

Preceding  Process 
Supply  Process 

Sub-process  1 :  Product  Supply 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
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Sub-process  10:  Progress  Against  Requirements 

Control  Process 

Sub-process  13:  Information  Dissemination 

Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 

Inputs 

•  Solicitation  (RFP,  SOW  or  SOO  with  Cost/Schedule  Requirements)  (SP  2) 

•  Specified  Requirements  (SP  19) 

•  Acquirer  Signed  Agreement  (contract  or  program  directive)  (SP  2) 

•  Approved  changes  (SP  13) 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Plans  and  schedules  trend  analysis  (SP  9) 

•  Requirements  trend  analysis  (SP  10) 

Entry  Criteria 

Inputs  have  been  approved  by  the  appropriate  agents 

•  Sponsor/User  Agreement 

•  Negotiated  Agreement 

•  Validated  Requirements 

Tasks 

The  developer  (as  acquirer)  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to 

consider  include  the  following: 

a)  Define  the  required  developer-supplier  relationships.  This  should  include  discussions  concerning 
all  work  and  products  to  be  delivered  against  the  technical  requirements.  This  should  include  audits 
and  review  of  the  processes.  This  should  include  a  Supplier  Performance  Management  Plan  to  be  sent 
to  Sub-process  2  for  inclusion  in  a  negotiated  agreement. 

b)  Participate  on  appropriate  supplier  product  teams.  The  effort  should  include  periodic  meetings  to 
verify  and  document  that  the  supplier  has  a  correct  and  complete  understanding  of  the  requirements 
and  processes  in  place  to  satisfy  them. 

c)  Monitor  supplier  performance  against  key  product  metrics.  A  detailed  list  of  all  Key  Product 
Metrics  (from  Sub-process  10  -  Progress  Against  Requirements)  should  be  provided  to  the  Supplier 
and  monitored  by  the  Acquirer. 

d)  Flow-down  changes  in  requirements  or  operational  concept  that  might  affect  the  supplier’s 
project.  An  accurate  Configuration  Management  (CM)  program  should  be  established  to  track  all 
requirements  and  changes  to  those  requirements  and  that  they  are  flowed  down  to  the  contractors  and 
sub-contractors. 

e)  Control  changes  to  requirements  made  by  the  supplier  that  would  affect  the  developer’s  project 
or  other  related  projects  or  products.  Any  changes  made  by  the  supplier  should  be  verified  against 
the  requirements  before  approval  of  such  changes.  Flow  down  and  control  changes  through  an  active 
Configuration  Management  program  and  report  to  Sub-process  12  -  Outcomes  Management). 

f)  Assess  supplier  performance  against  assigned  requirements  including  conduct  of,  or 
participation  in,  appropriate  technical  reviews.  The  acquirer  and  supplier  should  mutually  agree  on 
the  format  of  the  technical  reviews  and  how  to  resolve  misunderstandings,  oversites,  and  errors  (Sub¬ 
process  10  -  Progress  Against  Requirements). 

g)  Validate  products  delivered  from  the  supplier,  or  ensure  that  products  have  been  validated 
before  delivery  and  prior  to  integration  with  other  products  that  form  a  composite  end  product 


17 


Sub-process  3 


intended  to  meet  the  developer’s  specified  requirements.  This  is  a  critical  requirement  which 
requires  validation  of  all  work  and  products  delivered  as  early  in  the  process  as  practical,  to  ensure  that 
they  are  ready  when  needed  for  product  integration  and/or  for  Enabling  Products.  Validate  all  work 
and  products  delivered  (Sub-processes  32  and  Sub-process  33)  and  report  to  Sub-process  29  - 
Logical  Solution  Representations  Validation). 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Supplier  Performance  Management  Plan  (SP  2,  3) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

•  All  Key  Product  Metrics  have  been  successfully  accomplished 

•  All  Technical  Reviews  have  been  completed 

•  Delivered  Products  satisfy  requirements  and  approved  changes. 

Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Control  Process 

Sub-process  12:  Outcomes  Management 
Agents 

•  Acquirer/Developer 

•  Program  Executive  Officer  (PEO)  /  Program  Manager  (PM) 

•  User/Fleet 

•  Logistics 

•  Procurement 

•  Systems  Engineering 

Tools 

•  Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

•  Project  Management  Tools  (ex.  Microsoft  Project) 

•  Tools  Survey:  Requirements  Management  Tools 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Supplier  Agreement  Management  process 
areas 


Metrics  and  Measures 

•  Report  supplier  progress  against  Key  Product  Metrics 

•  Report  percentage  of  Flow  Down  requirements  changes  (CM) 

•  Report  on  percentage  of  products  delivered  that  have  been  validated  and  need  to  be  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  influence  the  Planning,  Assessment,  Control,  and  Implementation  Processes. 
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4.2  Technical  Management 


The  Technical  Management  Processes  are  to  be  used  to  plan,  assess,  and  control  the  technical  work  efforts 
required  to  satisfy  the  established  agreement.  The  relationship  of  the  three  Technical  Management  Processes 
for  planning,  assessing,  and  controlling  the  technical  effort  is  shown  in  Figure  4.2 

Acquisition  Documents ,  Agreement , 


Figure  4.2  -  Technical  Management  Processes 


NOTES 

1  The 

a) 

b) 

2  The 

a) 

b) 

3  The 


acquirer  can  be  either  one  of  the  following: 

Internal  to  the  enterprise  -  for  example,  another  project,  marketing  organization,  parent  project  of  a 
product  team  itself,  executive  manager,  supervisor. 

External  to  the  enterprise  -  for  example,  procurement  agency,  prime  contractor,  another  developer, 
buyer,  customer,  end  user,  owner,  purchaser, 
supplier  can  be  either  one  of  the  following: 

Internal  to  the  enterprise  -  for  example,  another  project,  functional  organization,  product  team. 
External  to  the  enterprise  -  for  example,  another  developer,  prime  contractor,  producer,  seller, 
subcontractor,  vendor. 

sub-processes  of  this  Guide  apply  to  the  developer  in  its  role  as  acquirer,  supplier,  or  both. 


4.2.1  Planning  Process 

This  process  is  used  to  support  enterprise  and  project  decision  making  and  to  prepare  necessary  technical  plans 
that  support  and  complement  project  plans  to:  (1)  arrive  at  a  decision  to  supply  services  according  to  an  external 
solicitation;  (2)  determine  whether  to  proceed  with  an  internal  enterprise  project  for  a  new  product  or  a  product 
improvement;  (3)  guide  the  work  efforts  that  will  meet  the  requirements  of  an  established  agreement;  or  (4) 
replan  applicable  processes  for  engineering  a  system.  Replanning  is  normally  initiated  (1)  when  required  by  an 
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agreement;  (2)  when  significant  variations  or  anomalies  are  identified  from  other  Technical  Management 
process  outcomes;  or  (3)  before  implementation  of  the  next  enterprise-based  life-cycle  phase. 

The  five  sub-processes  associated  with  the  Planning  Process  are  shown  in  Figure  4.2.1a. 


Planning 

Process 

Requirements 


Sub-process  4  -  Process  Implementation  Strategy 


Sub-process  5  -  Technical  Effort  Definition 


"Sub-process  6  -  Schedule  and  Organization 


ub-process  7  -  Technical  Plans 


ub-process  8  -  Work  Directives 


Figure  4.2.1a  -  Planning  Process/Sub-processes 


Sub-process  4  -  Process  Implementation  Strategy 

The  developer  shall  define  a  strategy  for  implementing  the  adopted  process  of  this  Guide  as  a  basis  for  project 
technical  planning  and  that  is  in  accordance  with  the  agreement. 


The  intent  is  to  provide  enough  information  for  the  user  to  determine  whether  a  given  process  activity  is 
appropriate  in  supporting  the  objectives  of  the  program  or  project  they  support  and  how  to  go  about 
implementing  the  process  activity. 

Note  that  the  act  of  planning  should  not  be  carried  out  in  a  vacuum.  It  is  iterative  and  thus  will  require  inputs 
regarding  the  Technical  Effort,  Schedule,  Technical  Plans,  and  Work  Directives. 

Preceding  Process 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 

Inputs 

Capability  Development  Document  (CDD)  or  Capability  Production  Document  (CPD)  -  formerly 
Operational  Requirements  Document  (ORD)  (SP  14) 

Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

This  is  where  it  all  starts.  When  someone  asks  the  simple  question,  “What’s  your  plan?”  or  “How  are  you  going 
to  get  it  done?”,  this  sub-process  is  initiated  and  the  whole  systems  engineering  process  is  begun.  It  is  reentered 
when  things  change  significantly,  such  as  funding,  requirements,  or  schedule. 

This  process  must  start  at  the  very  beginning  of  a  Major  Acquisition  at  Milestone  A  and  be  reviewed  at  each 
subsequent  Milestone  B,  C,  and  IOC.  An  example  of  when  you  may  reenter  this  process  would  be  when  a  Key 
Performance  Parameter  (KPP)  is  not  going  to  be  met,  requirements  change,  or  drastic  policy/funding/schedule 
changes. 

For  less  formal  projects,  the  entry  criteria  can  simply  be  a  request  from  a  Program  Manager  for  Systems 
Engineering  resources. 
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Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Identify  stakeholders  who  will  have  an  interest  or  stake  in  the  outcome  of  the  project.  Consider 
stakeholders  in  both  the  Funding  Chain  and  Beneficiary  Chain  (other  stakeholders,  primary  users,  etc.). 

b)  Identify  and  acquire  applicable  documents  and  the  requirements  therein,  that  could  affect  the 
project.  This  will  ensure  the  current  and  accurate  documentation  of  the  Engineering  Baseline.  The 
Systems  Engineer  is  responsible  for  the  implementation  of,  and  adherence  to,  approved  policies  and 
processes.  (For  NAVAIR,  reference  the  Class  Desk  Orientation:  Roles  &  Responsibities  presentation.) 
Making  the  applicable  documents  available  in  a  project  library  enables  the  project’s  personnel  to  easily 
access  the  same  baselined  information  as  they  perform  their  work.  At  a  minimum,  list  the  document 
name,  version,  and  date  for  historical  purposes.  This  information  should  be  stored  in  the  Enterprise 
Data  Repository  established  in  Sub-process  5. 

c)  Identify  process  approaches  required  to  develop  enabling  products  that  serve  as  the  roadmap  for 
program  execution  from  program  initiation  through  post-production  support.  The  essential 
elements  to  include  in  the  process,  but  not  to  be  limited  to,  are  risk  management,  training,  testing, 
modeling  and  simulation,  open  systems,  cost  as  an  independent  variable,  environment  considerations, 
and  source  of  support 

d)  Identify  applicable  enterprise-based  life-cycle  phases  (see  Appendix  B),  expected  work  product 
outputs,  applicable  management  reviews,  and  life-cycle-phase  exit  criteria.  This  SE  Guide  is 
defining  the  DoN  SE  activities  that  satisfy  the  DoD  5000  requirements. 

e)  Identify  and  define  how  the  applicable  processes  of  this  Guide  will  be  integrated,  how  internal 
and  external  projects  will  be  involved,  and  how  they  will  be  integrated. 

1)  Read  all  of  this  document  to  get  the  overall  interrelationship  of  the  processes  and  the  document’s 
philosophy  and  approach. 

2)  Take  into  account  the  phase  and  scope  of  your  program  using  the  available  documents  and  DoD 
5000,  if  required.  Do  this  early  in  a  program,  since  fewer  guiding  documents  will  be  available 
later  in  the  program. 

3)  Identify  an  initial  list  of  which  inputs  and  outputs  are  required  to  execute  the  program. 

4)  Tracing  the  inputs  and  outputs  through  sub-processes  will  reveal  a  number  of  things: 

a.  Determine  the  level  of  process  applicability  and  tailoring  required. 

b.  Additional  inputs  required. 

c.  Support  resources  required  and  where  these  resources  are  available. 

5)  Check  to  see  what  outputs  are  produced  by  each  process  to  see  if  all  apply  to  the  program 
considering  its  phase  and  scope.  The  descriptive  portion  of  the  tasks  of  a  sub-process  contains 
clarifications  of  these  outputs.  This  portion  also  gives  guidance  on  developing  the  output  by 
identifying  the  tools  and  organizations  that  are  involved,  and  detailing  some  interrelationships 
between  the  organizations. 

6)  Create  a  tailored  version  of  this  systems  engineering  process  for  your  project.  Creating  a  top-level 
plan  can  be  accomplished  by  developing  a  Gantt  chart  using  the  schedule  and  tasking  information 
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in  the  Inputs  and  the  tailored  process  list.  Consult  with  those  responsible  for  the  Technical  Effort, 
Schedule,  Technical  Plans,  and  Work  Directives  to  determine  how  the  details  will  be  filled  in. 


f)  Identify  and  define  progress  assessment  metrics  and  reporting  requirements.  The  frequency  and 
format  of  progress  reports  will  impact  the  effort  calculations  in  Sub-process  5  and  the  establishment  of 
schedules  in  Sub-process  6.  The  decision  whether  or  not  to  use  an  Earned  Value  Management  System 
will  also  have  impacts  in  Sub-processes  5  and  6. 

Select  meaningful  Metrics  and  Measures  specific  to  the  program  and  add  them  to  the  generic  list. 
Acknowledge  that  someone  else  is  responsible  for  executing  the  process.  That  person  will  be 
responsible  for  defining  and  collecting  metrics  for  both  the  process  itself  and  the  products  that  are 
produced.  Without  measuring  the  process  itself,  there  is  no  way  to  tell  that  a  change  to  the  process  was 
actually  an  improvement. 

g)  Prepare,  document,  and  make  available  the  process  implementation  strategy.  This  documentation 
should  also  include  details  for  modifications  to  the  process  implementation  strategy. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  List  of  stakeholders  and  roles  (SP  4,  15) 

•  Associated  process  approaches  (SP  4) 

•  Life-cycle  phase  chart  (Milestones)  (SP  4,  6,  8) 

•  Work  products  and  outputs  (SP  4)  -  (e.g.,  Work  Breakdown  Structure  (WBS)) 

•  Work  product  reviews  (SP  4) 

•  Life-cycle  phase  exit  criteria  (SP  4,  8) 

•  List  of  applicable  tasks  (SP  4) 

•  Program  metrics  and  reporting  requirements  (SP  4) 

•  Project  Library  (SP  5) 

•  Process  Implementation  Strategy  (SP  5,  6,  7,  8) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Planning  team  agrees  to  estimates  and  customers  acknowledge  receipt  of  information. 

Next  Processes 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Sub-process  8:  Work  Directives 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  15:  Other  Stakeholder  Requirements 

Agents 

Systems  Engineering 
Program  Manager 
Logistics 

Suggested  Tools 

Master  Acquisition  Planning  Program  (MAPP)  vl.l 
References 

Standard  across  all  systems  engineering  efforts: 
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•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Capability  Maturity  ModeKDIntegration  (CMMI— ),  2001:  Integrated  Project  Management  process  areas 

NAVAIR  Specific: 

•  Class  Desk  Orientation  Presentation,  March  2000 

•  NAVAIR  Acquisition  Guide,  January  2004 

•  NAVAIRINST  4200.36C,  Acquisition  Plans,  2004 

Metrics  and  Measures 
Estimated  cost  of  project 
Estimated  schedule  of  project 
Estimated  cost  and  time  spent  planning 
Actual  cost  and  time  spent  planning 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process:  (1)  provide  a  roadmap  for  the  technical  implementation  of  the  project, 
including  engineering  life-cycle  activities  within  specified  enterprise-based  life-cycle  phases;  (2)  are  to  be 
inplementable  by  each  product  team  or  product  manager;  (3)  are  used  in  preparing  and  negotiating  an 
agreement;  and  (4)  influence  the  developer’s  ability  to  fulfill  other  requirements  of  the  Planning  Process. 

The  process  implementation  strategy  includes  requirements  for  the  processes  to  be  undertaken,  applicable 
constraints,  completion  criteria,  and  feasibility  of  each  process,  considering  resources  (personnel,  materials,  and 
technology)  and  the  project  execution  environment.  This  strategy  can  be  a  part  of  the  project  plan  or  a  stand¬ 
alone  document. 


Sub-process  5  -  Technical  Effort  Definition 

The  developer  shall  define  a  technical  effort  that  is  in  accordance  with  the  process  implementation  strategy. 


Preceding  Process 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  4:  Process  Implementation  Strategy 
Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Sub-process  15:  Other  Stakeholder  Requirements 
Sub-process  16:  System  Technical  Requirements 


Inputs 

•  Process  Implementation  Strategy  (SP  4) 

•  Project  Library  (SP  4) 

•  Organizational  Structure  (SP  6) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Program  Operating  Guide  (POG)  (SP  6) 
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•  Acquirer  requirements  (SP  14) 

•  Measures  of  Effectiveness  (MOEs)  (SP  14) 

•  Other  stakeholder  requirements  (SP  15) 

•  System  technical  requirements  (SP  1 6) 

•  Data  and  Document  Management  Plans  (SP  7) 

•  Configuration  Management  Plans  (SP  7) 

•  Acquisition  Strategy  (SP  2) 

•  Cost,  schedule,  and  performance  constraints  (SP  2) 

•  Solicitation  (RFP,  SOW  or  SOO  with  Cost/Schedule  Requirements)  (SP  2) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Identify  project  tasks  to  include  all  requirements;  and  enterprise,  project,  and  associated  process 
constraints.  This  sub-process  will  address  what  tasks  an  organization  needs  to  do  to  define,  control, 
and  measure  its  work.  It  addresses  the  processes  and  not  the  products  or  the  results  of  the  work. 

Product  definition,  development,  tests  and  logistics  requirements  are  described  elsewhere  (sub¬ 
processes  14  through  19).  The  sub-process  produces  a  description  of  the  work  to  be  done,  resources, 
schedules,  funding,  and  reporting  requirements  for  competency  support.  (For  NAVAIR,  a  Team 
Assignment  Agreement  accomplishes  this  task.)  See  Sub-process  8  for  further  elaboration.  The 
Contract  defines  the  agreed  requirements  for  contracted  services.  See  Sub-processes  1  and  Sub¬ 
process  2  for  further  elaboration. 

The  ISO  9000  and  ISO  14000  family  of  management  system  standards  can  be  used  as  a  supplemental 
source  to  help  define  the  technical  tasks.  They  are  available  at  the  following  website: 
http://www.iso.ch/9000e/magical.htm.  The  management  system  standards  in  these  families  state 
requirements  that  the  organization  must  implement  to  manage  processes  influencing  quality  (ISO 
9000)  or  the  processes  influencing  the  impact  of  the  organization’s  activities  on  the  environment  (ISO 
14000).  Both  address  the  way  an  organization  defines  its  work,  and  not  directly  the  result  of  this  work. 

A  Technical  Data  Package  (TDP)  (reference:  http://www.nalda.navy.mil/techdata)  is  a  technical 
description  of  an  item  adequate  for  supporting  an  acquisition  strategy,  development,  manufacturing 
development,  production,  engineering,  and  logistics  throughout  the  item's  life  cycle.  The  TDP  should 
be  produced  as  part  of  the  data  that  makes  up  the  product  requirements.  This  sub-process  identifies  the 
need  for,  and  content  of,  the  TDP.  The  data  and  documentation  is  produced  by  Sub-process  7  and 
used  by  Sub-process  12.  Acquisition  programs  must  acquire  the  minimum  essential  data  required  to 
support  the  defense  system  life  cycle.  Timing  of  data  delivery  or  access  is  critical  to  support 
affordable  readiness. 

The  categories  of  data  that  may  be  included  in  a  TDP,  but  not  limited  to,  are: 

•  Product  Definition  Data:  This  denotes  the  totality  of  data  elements  required  to  completely 
define  a  product.  Product  definition  data  includes  geometry,  topology,  relationships, 
tolerances,  attributes,  and  features  necessary  to  completely  define  a  component  part  or  an 
assembly  of  parts  for  the  purpose  of  design,  analysis,  manufacture,  test,  and  inspection. 

•  Engineering  Drawings:  Engineering  drawings  disclose  the  physical  and  functional 
requirements  of  an  item  using  graphic  and/or  textual  presentations. 

•  Associated  Lists 

•  Specifications 

•  Standards 

•  Performance  Requirements 
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•  Quality  Assurance  Provisions 

•  Reliability  Data 

•  Packaging  Details 

•  Modeling  Data 

A  Technical  Data  Package  (TDP)  is  beneficial  in  supporting: 

•  Program  risk  assessment  and  design  management 

•  Evaluation  and  control  of  physical  and  functional  design  interrelationships  of  interdependent 
components,  equipment,  subsystems,  or  systems 

•  Configuration  management  and  configuration  control 

•  Re -procurement/Competition  in  Contracting  Act 

•  Competitive  procurement  of  the  system  or  sub-system 

•  Competitive  procurement  of  spares  and  repair  parts 

•  Standardization 

•  Training  personnel 

•  Installation  and  operation  of  items,  equipment,  subsystems,  or  systems 

•  Maintenance 

•  Overhaul,  repair,  and  rework 

•  Inspection  and  quality  control 

•  Cataloging  and  provisioning 

•  Logistics  operations  (i.e.,  demilitarization,  investigations,  etc.) 

•  Obsolescence  replacement 

b)  Establish  an  Enterprise  Data  Repository  (including  an  information  database)  that  will  allow 
capture  of  project  data  and  be  able  to  securely  retain  and  make  information  available,  as  required. 

After  this  repository  is  established  it  is  used  by  Sub-process  12  to  manage  the  outcomes  of  this 
process.  In  order  to  clarify  the  effort,  a  description  follows: 

Enterprise  Data  Repository.  An  information  repository  preserving  all  program  pertinent  information 
needed  by  any  and  all  of  the  program  stakeholders  should  be  established  and  maintained.  Information 
sharing  mechanisms  could  include  share  folders,  program  libraries,  formal  and  informal  presentations, 
technical  interchanges,  e-mail,  and  web  pages.  Appropriate  access  and  security  requirements  need  to 
be  defined  and  implemented.  It  should  at  least  contain  all  contract  relevant  documents,  program 
requirements,  position  papers,  official  communications,  risks,  action  items,  schedules  and  cost  data. 
This  repository  is  what  is  set  up  to  be  used  by  Sub-process  12  for  outcomes  management 

Common  References.  As  a  supplement  to  program  specific  information,  databases  and  repositories  of 
instructions,  MIL  STDs,  and  Industry  Standards  are  also  globally  available.  The  program  can  use 
these  to  more  thoroughly  define  its  technical  effort  in  a  disciplined  fashion  and  draw  on  a  large 
documented  source  of  expert  information.  Navy  and  Marine  Corps  Instructions  including  the  Design 
Reviews  Instruction  can  be  found  at  the  following  website:  http://www.naIda.navv.mil/instructions/. 
A  list  of  some  useful  MIL-STDs  can  be  found  in  Appendix  G:  Engineering  Specialty  References.  An 
index  of  on-line  standards  available  to  IEEE  subscribers  is  currently  at  the  following  website: 
http://standards.ieee.org/catalog/olis/index.html.  A  complete  listing  of  published  International 
Standards,  classified  by  subject,  is  available  at  the  following  website:  http ://www.iso.ch/.  The  AT&L 
Knowledge  Sharing  Systems  contains  mandatory  and  discretionary  policy  documents  including  laws, 
directives,  and  regulations.  It  can  be  found  at  the  following  website: 
http  ://deskbook.dau.mil/j  sp/default.  j  sp . 

c)  Determine  the  risk  management  strategy  to  identify  technical  risks  to  the  appropriate  level  and  to 
properly  avert  those  risks  that  could  adversely  affect  the  project.  Identify  the  effort  required  to  define 
and  control  technical  risks  that  need  to  be  considered  in  developing  a  Risk  Management  strategy.  Sub¬ 
process  7  will  address  what  needs  to  be  done  (planned)  to  implement  the  strategy  and  Sub-process  24 
discusses  the  risk  analysis  process.  In  order  to  define  the  effort,  a  definition  follows: 
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Risk  Management  is  the  systematic  approach  to  identifying,  analyzing,  and  controlling  areas  or  events 
with  a  potential  for  causing  unwanted  change.  It  is  through  risk  management  that  risks  to  the  program 
are  assessed  and  systematically  managed  to  reduce  it  to  an  acceptable  level.  Risk  is  a  measure  of  the 
inability  to  achieve  overall  program  objectives  within  defined  cost,  schedule,  and  technical  constraints. 
It  has  two  primary  components:  (1)  the  probability  of  failing  to  achieve  a  particular  outcome  and  (2) 
the  consequences  of  failing  to  achieve  that  outcome.  Risk  Management  is  the  act  or  practice  of 
controlling  risk.  The  Risk  management  strategy  must  include  risk  planning,  assessing  risk  areas, 
developing  risk-handling  options,  monitoring  risks  to  determine  how  risks  have  changed,  and 
documenting  the  overall  risk  management  program.  The  requirements  of  the  Risk  Advisory  Board 
should  be  developed. 

One  source  of  cross-discipline  information  for  the  items  that  need  to  be  considered  in  developing  a 
Risk  Management  Plan  and  Strategy  is  in  the  following  document:  Top  Eleven  Ways  to  Manage 
Technical  Risk.  It  is  found  on  the  web  at:  http://www.abm.rda.hq.navy.mil/p3686.pdf .  It  offers 
concise  explanations  and  clear  descriptions  of  steps  one  can  take  to  establish  and  implement  core 
technical  risk  management  functions.  It  contains  basic  information,  explanations,  and  best  practices. 

It  also  contains  the  Risk  Management  requirements  from  DoD  Directive  5000.1  and  DoD  Instruction 
5000.2,  which  provide  the  mandatory  policies  and  procedures  for  the  management  of  acquisition 
programs. 

The  Department  of  the  Air  Force  Software  Technology  Center’s  Guidelines  for  Successful  Acquisition 
and  Management  of  Software-Intensive  Systems,  Chapter  6,  also  provides  another  good  resource  for 
addressing  risk  management.  It  builds  on  the  premise  that  effective  risk  management  depends  on  the 
successful  integration  of  both  the  supplier  and  buyer’s  risk  management  processes. 

Additional  information  is  available  at  DoD  websites  such  as: 

http://www.acq.osd.mil/io/se/risk  management/index.htm 

http://www.acq.osd.mil/io/se/ 

d)  Define  product  metrics  by  which  the  quality  of  the  products  will  be  evaluated  and  process  metrics 
by  which  the  efficiency  and  effectiveness  of  the  technical  effort  will  be  measured.  The  following 
program  metrics  should  be  collected  and  analyzed,  as  a  minimum,  in  order  to  determine  trends; 
performance  strengths  and  weaknesses;  and  probability  of  success. 

Program  Metrics: 

Cost:  Projected  and  Actual  Expenditures  (BCWP,  ACWP,  BCWS) 

Schedule  Compliance:  Time  allotted  and  taken,  variance 
Performance:  Requirements  met,  not  met,  or  deferred 
Risks:  number  and  severity 

Critical  Path:  Number  of  Items  along,  Performance  along 
Divergence  from  historical  programs:  Novelty,  State-of-the-Art 
External  Dependencies 
Staffing 

Product  Metrics: 

Measures  of  Effectiveness  (MOEs)  Achievement 
Achievement  of  Key  Performance  Parameters  (KPPs) 

Technical  Performance  Measures  (TPMs) 

Complexity/Producibility 
Requirements  Traceability 

Requirements  and  Design  Changes:  Change  Requests  pending 

Quality  and  Stability:  System  Trouble  Reports  pending,  Trend  Analysis,  Rework 

Computer  Resource  Utilization 

Software  Metrics:  AVDEP-HDBK-7,  Rev.l,  dated  1  Feb  1996  -  Software  Metrics  Program 
addresses  requirements,  size,  staffing,  quality,  capacity,  and  schedule  metrics 
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Testing  Metrics:  Requirements  identified,  tested  and  passed 


Process  Metrics:  Capability  Maturity,  Statistical  Process  Control  (SPC) 

The  Carnegie  Mellon  Software  Engineering  Institute  website  at  http://www.sei.cmu.edu/  has 
additional  information. 

e)  Establish  cost  objectives  (e.g.,  ownership,  acquisition,  operating,  support,  and  disposal)  to  be  used  in 
Trade-off  analyses. 

Cost  as  an  Independent  Variable  (CAIV).  As  part  of  the  DoD  Acquisition  Reform  Initiative  to 
quantify  and  manage  Total  Ownership  Costs  (TOC),  CAIV  methodology  must  be  established  and 
utilized  throughout  the  entire  life  cycle  of  the  acquisition  process  to  ensure  that  operational  capability 
of  the  total  force  is  maximized  for  a  given  investment.  CAIV  methodology  entails  the  consideration  of 
cost  along  with  required  system  capabilities;  cost  is  neither  dominant  nor  dependent,  but  rather  a  peer 
with  other  characteristics.  Cost  will  be  formally  considered  for  all  Milestones  after  MS  0  by 
conducting/updating  an  analysis  that  relates  cost  and  all  system  capabilities  to  the  system’s  battlefield 
contribution.  This  approach  is  not  independent  of  all  work  to  determine  specific  capabilities,  rather  it 
is  part  of  it.  Cost  performance  analyses  will  be  conducted  on  a  continuous  basis  throughout  the  life 
cycle. 

1)  CAIV  will  be  applied  to  AC  AT  I,  II,  III  programs.  AC  AT  IV  programs  shall  use  CAIV  as  a 
guideline. 

2)  PEOs  and  PMs  shall  plan  for  the  conduct  of  cost-performance  trade-off  studies. 

3)  Aggressive  cost  targets  for  development,  procurement,  Operations  and  Support  (O&S)  and 
disposal  must  be  established  at  each  milestone  review.  Progress  for  achieving  cost  targets  shall  be 
presented  at  each  milestone  review. 

4)  Cost-performance  objectives  and  cost  targets  shall  be  included  in  procurement  documents  and 
contractor  statements-of-work,  as  appropriate. 

Post  Deployment  Costs.  Life  Cycle  Management  Plans  and  In-Service  Engineering  Agent  (ISEA) 
plans  should  be  developed  to  address  post  deployment  ownership,  operating,  support,  and  disposal 
strategies  and  costs. 

f)  Identify  technical  performance  measures  that  will  be  used  to  determine  the  success  of  the  system,  or 
portion  thereof,  and  that  will  receive  management  focus  and  be  tracked  using  Technical 
Performance  Measurement  (TPM)  procedures.  This  would  include  incremental  measures  taken  to 
assess  the  probability  of  meeting  the  objectives.  It  could  include  specific  measures  to  determine 
reliability,  maintainability,  availability,  survivability,  testability,  safety,  electromagnetic  properties, 
weight,  balance,  and  manufacturability.  TPMs  are  derived  from  MOPs,  which  reflect  system 
requirements.  MOPs  are  derived  from  MOEs,  which  reflect  operational  requirements.  Sub-process 
16  task  c)  identifies  the  KPPs. 


NOTE:  A  TPM  program  provides  an  early  warning  of  the  adequacy  of  a  design  in  terms  of 
satisfying  selected  key  performance  parameter  requirements  of  a  system  end  product.  TPM  also 
examines  marginal  cost  benefit  of  performance  in  excess  of  requirements.  It  also  includes 
sensitivity  analysis.  A  Key  Performance  Parameter  (KPP)  is  one  that  characterizes  a 
significant  total  system  qualifier.  In  addition  it  must  be  possible  to  project  the  evolution  of  the 
parameter  as  a  function  of  time  toward  the  desired  value  at  the  completion  of  development.  The 
projection  can  be  based  on  verification  validation  planning  or  historical  data. 
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g)  Identify  applicable  tasks  based  on  analysis  of  the  key  events  of  the  project,  and  the  entry  and  exit 
criteria  for  each  event. 

Work  Breakdown  Structure  (WBS)  is  the  mechanism  used  to  display  and  define  the  product  to  be 
developed  or  produced  by  hardware,  software,  support,  and/or  service  element,  and  relates  the  work 
scope  elements  to  each  other  and  to  the  end  product(s).  It  also  defines  all  contractual  authorized  work. 
The  WBS  Dictionary  is  an  important  aspect  of  the  work  breakdown  structure  and  should  be  given 
appropriate  attention  in  development  of  the  WBS.  After  Contract  award,  the  Project  Manager  expands 
the  WBS  into  a  Contract  Work  Breakdown  Structure  (CWBS)  as  the  initial  step  in  the  planning 
process.  WBS  expansion  will  extend  the  CWBS  a  minimum  of  one  level  below  the  negotiated  external 
reporting  level.  This  sets  up  the  framework  for  work  scope  definitions  and  assignments  to  the 
functional  organizations  responsible  for  performing  the  work.  The  CWBS  is  used  internally  to  plan  the 
program  in  detail  and  to  collect  status  information  on  a  periodic  basis.  The  adequate  number  of  levels 
of  each  CWBS  leg  extension  is  determined  by  the  contractual  work  scope,  level,  EVMS  requirements 
using  the  negotiated  Cost  Performance  Reports  (CPR)  or  Cost/Schedule  Status  Reports  (C/S SR)  and 
the  Project  Manager’s  management  style.  The  CWBS  is  not  a  ’’people"  organization  chart;  it  is  a  work 
scope  chart. 

For  Government  contracts,  use  MIL-HDBK-881  (latest  revision)  as  a  WBS  design  guide.  MIL- 
HDBK-881;  DoD  Handbook  —  Work  Breakdown  Structure;  2  January  1998  is  approved  for  use  by  all 
Departments  and  Agencies  of  the  Department  of  Defense  as  guidance,  although  it  cannot  be  cited  as  a 
requirement.  The  handbook  addresses  mandatory  procedures  for  those  programs  subject  to  DoD 
Regulation  5000. 2-R.  It  also  provides  guidance  to  industry  in  extending  contract  work  breakdown 
structures. 

Earned  Value.  In  order  to  objectively  define  the  program  baseline  cost  objectives  and  track  them 
against  performance  and  schedule,  an  Earned  Value  Management  System  must  be  established.  Earned 
Value  is  a  management  technique  used  to  integrate  cost,  schedule,  technical  performance  management, 
and  risk  management.  EVM  System  Industry  Standards  (ANSI/EIA-748-1998)  Section  2  (defined  in 
the  Interim  Defense  Acquisition  Guidebook,  formerly  DoD  5000. 2-R)  contains  the  32  EVMS 
Guidelines  that  should  be  applied.  It  mirrors  the  DoD  Earned  Value  Management  Implementation 
Guide  (EVMIG). 

Inputs  to  Earned  Value  require  the  project  manager  to  plan,  budget  and  schedule  the  authorized  work 
scope  (as  defined  in  the  WBS)  in  a  time-phased  plan.  As  work  is  accomplished,  it  is  “earned”.  Earned 
Value  compared  with  planned  value  provides  a  work  accomplished  against  plan.  A  variance  to  the 
plan  is  noted  as  a  schedule  or  cost  deviation.  Normally  the  established  accounting  system  provides 
accumulation  of  actual  cost  for  the  project.  The  actual  cost  is  compared  with  the  earned  value  to 
indicate  an  over  or  under  run  condition.  Planned  Value,  Earned  Value,  and  Actual  Cost  data  provides 
an  objective  measurement  of  performance,  enabling  trend  analysis  and  evaluation  of  cost  estimate  at 
completion  within  multiple  levels  of  the  project.  Through  disciplined  use  of  systematic  processes, 
programs  are  expected  to  integrate  contract  work  scope,  budget,  and  schedule  to  achieve  a  realistic, 
executable  contract  plan  called  the  Performance  Measurement  Baseline  (PMB).  EVM  learning 
opportunities  are  an  integral  part  of  the  various  Defense  Acquisition  University  (DAU)  short  courses, 
as  well  as  the  flagship  Representative  for  Engineering  (NAV AIR’s  APMSE)  course. 

Scheduling.  This  is  a  key  element  of  the  EVMS  system,  which  addresses  the  time  dependency  of  the 
acquisition  process.  The  detailed  schedule  and  organization  chart  based  on  EVMS  is  produced  in  sub¬ 
process  6.  Some  parameters  that  should  be  considered  when  developing  a  schedule  to  support  a 
successful  EVMS  process  include  Accuracy,  Reliability,  Simplicity,  Universality  (sufficient  from 
beginning  to  end  of  a  project),  Decision  Analysis  (enables  management  to  simulate  the  impact  of 
alternative  courses  of  action),  Forecasting,  Updating,  Flexibility,  and  Cost.  Examples  of  Scheduling 
Techniques  include:  Flow  charts,  Leadtime  charts  or  Set-back  charts,  Milestone  Charts,  Bar  Charts, 
Gantt  Chart,  Modified  Gantt  /  Milestone  charts,  Critical  Path  Method  (CPM),  Directed  Date  and 
PERT. 
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h)  Identify  the  appropriate  methods  and  tools,  required  facilities  and  equipment,  and  training 
required  to  be  able  to  complete  defined  tasks  and  meet  event  exit  criteria. 

Facilities  and  Equipment.  The  land,  air  and  sea  facilities,  laboratories,  special  fixtures,  simulators, 
and  Test  Ranges  required  during  the  total  life  cycle  of  the  program  must  be  identified,  funded, 
scheduled,  developed  and/or  procured.  Facilities,  Laboratories  and  Ranges  should  be  treated  as  an 
integral  part  of  the  program  planning  process.  In  addition  to  traditional  development  and  life-cycle 
support  labs,  this  could  include  wind  tunnels,  anechoic  chambers,  and  EMI  facilities.  The  location  at 
which  the  system  is  finally  deployed  and/or  operationally  tested  may  be  a  consideration  parameter. 

Tools.  The  following  International  Council  on  Systems  Engineering  (INCOSE)  website  serves  as  an 
excellent  reference  for  identifying  the  various  types  of  tools: 

http://www.incose.org/tools/tooltax/se_tools_taxonomy.html 

A  summary  outline  of  that  information  follows: 

SE  Tools  Taxonomy  -  Management  Tools 

Configuration  Management  Tools 
Work  Flow  Management  Tools 
Risk  Management  Tools 
Cost  Estimation  and  Tracking  Tools 
Cost  Estimation  Tools 
Cost  Tracking  Tools 
Defect  Tracking  Tools 

SE  Tools  Taxonomy  -  Engineering  Tools 

System  Design  Tools 

System  Model  Tools 

Structural  Modeling  Tools 
Behavioral  Modeling  Tools 

Static  Behavioral  Tools 
Dynamic  Behavioral  Tools 
HMI  Prototyping 
Design  Support  Tools 

Simulation  Tools 
Numerical  Analysis  Tools 
Domain  Specific  Tools 
Measures  of  Effectiveness  Tools 

Requirements  Engineering  Tools 

Requirements  Management  Tools 

Requirements  Classification  Tools 
Requirements  Capture  &  Identification  Tools 
Textural  Requirements  Capture  Tools 
Tools  for  Elicitation  of  Requirements 
Requirements  Traceability  Tools 
Requirements  Generation  Tools 

Design  Validation  Tools 

Threat  Analysis  Tools 

Test  Validation  Planning  Tools 

Scenario  Validation  Tools 

Tools  to  Validate  System  Compliance  with  Requirements 
Measurement  Tools 
Performance  Analysis  Tools 
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Specialty  Engineering  Tools 


SE  Tools  Taxonomy  -  Information  Sharing  Tools 

Communication  Tools 

Interpersonal  Communications  Tools 
Network  Information  Retrieval  Tools 
Data  Analysis  Tools 

Spreadsheet  Tools 
Data  Reduction  Tools 
Data  Visualization  Tools 

Electronic  Publishing  Tools 
Electronic  Viewing  Tools 
Tool  Integration  Facilities 

SE  Tools  Taxonomy  -  Infrastructure  Support  Tools 

System  Administration  Tools 
Network  Support  Tools 
Product  Data  Management 

i)  Identify  applicable  or  potential  technology  constraints  and  develop  an  approach  for  overcoming 
each  constraint,  by  using  an  appropriate  mitigation  approach  and  by  technology  insertion  at  the 
appropriate  time  in  the  enterprise-based  life  cycle. 

Identify  constraints  on  the  system  including: 

•  Analysis  of  Alternatives  (AO  A):  The  AO  A  should  assess  the  critical  technologies  associated  with 
the  system  development  concepts,  including  technology  maturity  and  technical  risk. 

•  Use  of  Commercial  Off  The  Shelf  (COTS)  equipment 

•  Use  of  Non-Development  Items  (NDI) 

•  Use  of  Existing  Facilities 

Functional  and  performance  requirements  must  be  compared  with  existing  technologies  to  ascertain 
feasibility  of  accomplishment.  Any  functional  or  performance  constraints  imposed  by  existing 
technology  must  be  identified.  If  at  this  early  stage  it  is  known  that  new  technology  must  be 
developed,  a  summary  of  the  development  status  should  be  provided.  From  this  status,  technical  risk 
and  cost  should  be  estimated. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Technical  Data  Package  (TDP)  (SP  16) 

•  Enterprise  Data  Repository  (SP  12) 

•  Risk  Management  Strategy  (including  Risk  Advisory  Board  requirements)  (SP  24) 

•  Program  metrics  (SP  9) 

•  Process  metrics  (SP  9) 

•  Product  metrics  (SP  10) 

•  Testing  metrics  (SP  11) 

•  CAIV  decision  criteria  (SP  22) 

•  Total  Fife  Cycle  Cost  Objectives  (SP  6,  8) 

•  Technical  Performance  Measures  (TPM)  (SP  9,  10,  11,  22) 

•  Work  Breakdown  Structure  (WBS)  (with  WBS  Dictionary)  (SP  2,  6,  7,  9,  10) 

•  Inputs  to  Earned  Value  Management  System  (EVMS)  (SP  8,  9) 

•  Technology  Roadmap  (SP  16) 

•  Fist  of:  Methods  and  Tools,  Facilities,  Equipment,  Training  (SP  32) 
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Life  Cycle  Support  Plans  (SP  16) 

Pre-Plan  Product  Improvement  (P3I)  (SP  16) 


Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

•  Total  Ownership  Cost  established 

•  Risk  Management  Strategy  defined 

•  EVMS  Requirements  established 

•  Metrics  identified 

•  Information  repository  set  up 

•  Methods,  tools,  training  and  facilities  identified 

Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Sub-process  8:  Work  Directives 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  24:  Risk  Analysis 
System  Verification  Process 

Sub-process  32:  Enabling  Products  Readiness 

Agents 

Acquirer:  PEO/PM 
End  User 

Systems  Engineering 
Technical  Writer 

Tools 

WBS  Instructions  and  Plan,  EVMS  Instructions  and  Plan 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  Defense  Acquisition  Deskbook 

•  FAR/DFARs 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Top  Eleven  Ways  to  Manage  Risk,  ASN/RD&A,  October  1998 
Capability  Maturity  Model®  Integration  (CMMI)sm,  http://www.sei.cmu.edu/ 

(especially  General  Information,  Organizational  Innovation  and  Deployment  process  area,  and  Measurement 
and  Analysis  process  area) 
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Business  Case  Analysis  Risk  Assessment  Matrix 

Managing  Quality  and  Productivity  in  Aerospace  and  Defense,  November  1989 
OSD  Commercial  Item  Acquisition:  Considerations  and  Lessons  Learned,  26  June  2000 
AVDEP-HDBK-7,  Rev.l,  dated  1  February  1996  -  Software  Metrics  Program 

EIA,  Earned  Value  Management  Systems  (EVMS),  (EIA-748),  1998 

MIL-HDBK-881,  Work  Breakdown  Structure;  2  January  1998 

ISO  9000 
ISO  14000 

The  Department  of  the  Air  Force  Software  Technology  Center’s  Guidelines  for  Successful  Acquisition  and 
Management  of  Software-Intensive  Systems,  Chapter  6 

NAVAIR  Specific: 

•  Draft  NAVAIR  Risk  Management  Guide 

•  DAU  Earned  Value  Management  Department:  http://www.dsmc.dsm.mil/educdept/evm_dept.htm 

•  Earned  Value  Management  System  Policy:  http://www.dcmc.hq.dla.mi1/onebook/2.0/2.2/EVM.htm 

•  Earned  Value  Management  Implementation  Guide: 
http://www.acq.osd.mil/pm/currentpolicy/iig/evmigLhtm 

Metrics  and  Measures 

Risk  Cube 

EVMS 

WBS 

Capability  Maturity 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  will  provide  guidance  for  preparing  schedules  and  applicable  technical  plans 
and  for  identifying  resource  requirements,  and  will  influence  the  developer’s  ability  to  complete  the  other 
applicable  processes  for  engineering  a  system. 


Sub-process  6  -  Schedule  and  Organization 

The  developer  shall  schedule  and  organize  the  defined  technical  effort. 


Provide  a  task-oriented  sequence  of  events  and  resources  that  serves  as  the  roadmap  for  meeting  the  plans, 
objectives  and  milestones  of  the  customer. 

Preceding  Process 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  4:  Process  Implementation  Strategy 
Sub-process  5:  Technical  Effort  Definition 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 

Inputs 

•  Acquisition  strategy  (SP  2) 

•  Total  Life  Cycle  Cost  Objective  (SP  5) 

•  System  Technical  Requirements  (SP  16) 

•  Work  breakdown  structure  (WBS)  (SP  5) 

•  Life  Cycle  Phase  Chart  (Milestones)  (SP  4) 

•  Process  Implementation  Strategy  (SP  4) 
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Interim  Defense  Acquisition  Guidebook  (formerly  DoD  5000-2R)  requires  all  major  acquisition  programs 
(AC AT  I  and  II)  to  have  an  Acquisition  Program  Baseline  (APB).  This  document  contains  key  milestones  and 
events  for  the  program  (i.e.,  MS-A  (Technology  Development),  MS-B  (System  Development  and 
Demonstration  (SDD)),  MS-C  (Production  and  Deployment),  Initial  Operational  Capability  (IOC),  etc.). 
However,  be  aware  that  for  most  programs  this  document  doesn’t  exist  so  other  means  must  be  used  to  obtain 
similar  data  that  is  expressed  in  the  APB  document.  For  most  programs,  important  schedule  information  is 
provided  by  the  sponsor  (acquirer)  or  program  office  through  formal  and  informal  channels.  We  recommend 
that  this  information  be  provided  through  formal  channels. 

All  programs  have  constraints  that  must  be  known  at  the  time  of  inception.  The  SE  has  to  know  and  understand 
the  cost,  schedule,  and  performance  constraints  and  thresholds.  These  constraints  must  be  known  before  any 
realistic  schedule  can  be  developed.  This  information  should  be  discussed  with  the  system  acquirer  and 
formally  documented  and  communicated  to  the  team. 

System  Technical  Requirements  serve  as  the  basis  for  scheduling  technical  activities.  Knowing  the  technical 
requirements  can  help  in  analyzing  various  schedule  options  and  lead  times  associated  with  activities  required 
to  deliver  the  system  solution.  For  example,  hardware  solutions  may  require  different  activities,  skills  and 
schedules  than  software  solutions.  A  Helicopter  aircraft  solution  may  require  different  activities  and  solutions 
than  a  fixed-wing  aircraft. 

A  properly  prepared  WBS  serves  as  a  good  top-level  source  for  identifying  what  needs  to  be  done  for  the  entire 
program.  A  schedule  should  be  identified  for  each  element  of  the  WBS.  The  level  of  the  WBS  normally 
dictates  the  level  of  schedule  information  that  will  be  tracked  in  a  common  database  within  the  program.  Lower 
level  elements  are  normally  tracked  at  the  lower  element  level  WBS. 

Milestones  serve  as  a  metric  of  progress  and  also  normally  identify  a  decision  point  for  management.  The 
Acquirer  normally  identifies  program  milestones,  and  the  SE  identifies  technical  milestones  within  the  scope  of 
the  program  milestones.  Most  programs  have  a  milestone  for  program  go-ahead,  contract  award  or,  in  the  case 
of  a  field  activity,  issuance  of  a  task  statement,  test  milestones  (DT  &  OT),  initial  operational  capability  (IOC) 
and  production  milestones. 

Entry  Criteria 

Inputs  have  been  approved  by  the  appropriate  agent. 

•  Milestone  approval 

•  Receipt  of  funding 

•  Request  from  Acquirer 

Tasks 

The  developer  should  plan  to  do  the  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include 
the  following: 

a)  Develop  an  event-based  schedule  that  integrates  key  events  (internal  and  external),  related  tasks, 
specialty  engineering  tasks,  and  relevant  completion  criteria  for  the  applicable  enterprise-based 
life-cycle  phase.  This  task  is  accomplished  based  on  the  scope  and  definition  of  the  technical  effort 
identified  in  Sub-process  5.  Navy,  prime  contractor,  and  sub-contractors  must  generate  definition  of 
tasks  and  responsibilities.  These  organizations  must  sign-up  to  produce  the  products  contracted  for,  to 
be  accountable  to  the  next  higher  level  of  product  development  and  integration,  and  to  support  the 
integration  of  their  product  as  part  of  the  total  system  integration.  This  assignment  of  tasks  and 
responsibilities  completes  the  development  of  the  WBS  initiated  under  Sub-process  5. 

b)  Develop  the  calendar-based  schedule,  showing  the  dates  of  expected  task  and  event  completions 
and  the  dependency  relationships  among  tasks,  with  the  goal  of  developing  information  for  an 
Integrated  Master  Schedule  (IMS).  The  IMS  is  the  integrated  schedule  of  the  program.  It  is  used  for 
identification  of  problem  areas  during  program  planning  and  execution  and  to  help  define  priorities  for 
management  attention  and  action,  particularly  as  problem  areas  develop  and  are  identified.  As  changes 
appear  to  be  required,  the  schedule  is  used  as  a  basis  for  evaluating  changes  and  is  a  significant  tool  for 
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communicating  the  program  content,  workflow,  and  approach.  Since  progress  can  be  compared  to 
planned  progress,  the  schedule  is  a  key  ingredient  to  providing  performance  measurement  and 
evaluating  remaining  work  scope  and  duration. 

The  IMS  is  the  tool  that  provides  the  detailed  tasks  and  timing  of  the  tasks  that  support  the  work  effort 
the  Integrated  Master  Plan  (IMP)  delineates.  It  supports  all  the  criteria,  accomplishments,  and  events 
of  the  IMP.  It  includes  process  tasks  as  required  to  insure  the  fully  integrated  plan  for  the  content  of 
the  program.  The  IMS  ties  them  together  by  showing  their  logical  relationships,  any  interrelationships 
between  pieces  of  work,  and  any  constraints  that  control  the  start  or  finish  of  each  piece  of  work. 
Thereby,  the  IMS  becomes  the  source  that  depicts  the  planned  dates  when  each  event  is  expected  to 
occur  as  well  as  all  the  expected  dates  for  all  work  done  to  get  to  the  event.  We  recommend  that  you 
use  software  tools  to  track  and  show  dependency  relationships.  These  tools  offer  the  user  the 
advantage  of  quickly  performing  changes  and  sensitivity  analysis. 

c)  Identify  resources  required  to  complete  scheduled  tasks.  Skills,  man-years,  and  cost  should  be 
identified  and  considered  in  the  overall  schedule  and  organization  of  each  phase  of  the  program. 
Facilities  such  as  national  assets  and  unique  assets  must  be  considered  in  the  schedule  along  with  key 
events. 

d)  Define  the  staffing  and  discipline  needs  to  complete  the  scheduled  tasks,  training  needs,  and  risks 
if  required  staff  are  not  available.  Staffing  is  very  often  forgotten  but  is  a  key  component  to 
successful  and  complete  scheduling  of  program  tasks.  The  systems  engineer  should  understand  the 
scope  of  the  technical  effort  identified  in  Sub-process  5  and  identify  the  staffing  required  for  program 
success.  The  systems  engineer  should  properly  phase  the  technical  staffing  needs  of  the  program. 
Consideration  should  be  given  to  availability  of  expertise  to  coincide  with  the  program  technical  effort 
needs.  Appropriate  subject  matter  experts  cannot  always  be  made  available  based  on  demand  and 
location  of  limited  resources  (funding  resources  or  human  resources).  Staffing  may  drive  schedule  or 
schedule  may  drive  staffing  depending  upon  resources  available. 

e)  Define  the  team  and  organizational  structure  to  complete  the  scheduled  tasks  within  resource 
constraints.  For  NAVAIR,  this  is  the  Program  Operating  Guide  (POG),  which  should  exist  for  most 
major  programs.  This  guide  is  useful  in  providing  the  IPT  structure,  key  individuals  and  support 
activities,  resources  and  man-year  information,  and  also  program  timelines  and  events  that  are 
important  for  a  twelve-month  period.  The  POG  is  used  to  lay  out  a  Program  Manager’s  plan  and 
guidance  for  the  concept  of  Integrated  Program  Teams  (IPTs)  and  their  relationship  to  the  Competency 
Aligned  Organization,  and  to  clearly  communicate  the  Program’s  organizational  structure  to  the 
Program’s  workforce,  the  Program  Executive  Officer  (PEO)  and  the  NAVAIR  leadership.  It  identifies 
the  goals,  objectives  and  attributes  of  the  team  and  is  updated  on  an  as  required  basis. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Resource  requirements  (staffing,  cost)  (SP  7) 

•  Organizational  structure  (SP  5,  7,  8) 

•  Integrated  Master  Schedule  (IMS)  (SP  2,  5,  7,  8,  9,  10,  11,21) 
NAVAIR  Specific: 

•  Program  Operating  Guide  (POG)  (SP  5) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

•  All  tasks  and  work  allocated  plus  resources  identified. 

•  Firm  organizational  structure 

Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
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Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  7:  Technical  plans 
Sub-process  8:  Work  Directives 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Transition  to  Use  Process 

Sub-process  21:  Transition  to  Use 

Agents 

Acquirer,  Systems  Engineering,  User,  Specialty  Engineering,  Logistics 
Tools 

Scheduling  Tools  (ex.  MS  Project,  Open  Plan,  Simplicity,  Primavera) 

Estimating  Tools  (ex.  COCOMO,  SEER-SEM,  Function  Points) 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

DoD  5002.  R  (Cl  .4  acquisition  program  baseline;  C5.2  Systems  engineering;  AP4  EVMS; 
C6.1Test  &  evaluation) 

•  Defense  Acquisition  Deskbook 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

DI-MISC-81183A  Integrated  Master  Schedule  Data  Item  Description  (DID) 

Capability  Maturity  Model®  Integration  (CMMI-),  2001:  Project  Planning  and  Integrated  Project 
Management  process  areas 

Metrics  and  Measures 
Schedule  variance  (SV) 

Cost  variance  (CV) 

Staffing 

Percent  complete 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  guidance  for  preparing  applicable  technical  plans  used  to  guide 
completion  of  the  technical  efforts  for  each  applicable  process  to  meet  agreement  requirements. 


Sub-process  7  -  Technical  Plans 

The  developer  shall  create  technical  plans  to  ensure  an  integrated  and  cost  effective  technical  effort  in 
accordance  with  the  defined  schedule  and  organization. 


Preceding  Process 
Planning  Process 

Sub-process  4:  Process  Implementation  Strategy 
Sub-process  5:  Technical  Effort  Definition 
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Sub-process  6:  Schedule  and  Organization 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 

Inputs 

•  Capability  Development  Document  (CDD)  or  Capability  Production  Document  (CPD) 

-  formerly  Operational  Requirements  Document  (ORD)  (SP  14) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Measures  of  Effectiveness  (MOE)  (SP  14) 

•  Process  Implementation  Strategy  (SP  4) 

•  Work  Breakdown  Structure  (WBS)  (SP  5) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Organizational  Structure  (SP  6) 

•  Resource  Requirements  (SP  6) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

•  Key  milestones  established. 

•  Technical  effort  and  organization  defined. 

Tasks 

The  developer  should  prepare  appropriate  plans  to  complete  this  sub-process.  Systems  engineering 
planning  addresses  the  scope  of  the  technical  effort  required  to  develop  the  system.  The  basic  questions  of 
“who  will  do  what”  and  “when”  must  be  answered.  A  technical  plan  describes  what  must  be  accomplished, 
how  systems  engineering  will  be  done,  how  the  effort  will  be  scheduled,  what  resources  will  be  needed,  and 
how  the  effort  will  be  monitored  and  controlled.  The  number  and  type  of  plans  will  vary  depending  on  the 
scope,  life-cycle  phase,  and  other  factors.  Appendix  D  of  this  document  contains  a  list  of  typical  technical 
plans.  Plans  to  consider  include  the  following: 

a)  Engineering  Plan.  For  most  Navy  and  Marine  Corps  programs,  this  implies  a  Systems  Engineering 
Plan  (SEP),  formerly  Systems  Engineering  Management  Plan  (SEMP).  On  major  programs  the  SEP  is 
a  contract  deliverable  and  is  prepared  by  the  prime  contractor.  Guidance  on  the  content  and  format  of 
a  SEP  (e.g.,  SEMP)  can  be  found  in  the  DAU  publication  “Systems  Engineering  Fundamentals”  and  in 
the  APMSE  Quick  Reference  Guide.  Also  see  the  list  of  questions  in  Table  C.7  for  ideas  on  what 
information  the  SEP  needs  to  provide.  Another  source  of  guidance  is  DI-MGMT -8 1 024 . 

The  Software  Development  Plan  (SDP)  is  the  equivalent  of  a  SEP  when  the  system  under  development 
is  purely  software  and  for  the  software  component  of  a  system.  Guidance  on  the  content  and  format  of 
an  SDP  can  be  found  in  ISO/IEC  12207.  On  programs  that  are  procuring  software  intensive  systems, 
the  planning  information  should  be  incorporated  into  the  corresponding  sections  of  documents  such  as 
the  Acquisition  Plan  and  the  SEP. 

b)  Risk  Management  Plan.  The  development  of  the  Risk  Management  Plan  supports  Sub-process  24, 
Risk  Analysis,  and  is  based  on  the  Risk  Management  Strategy  developed  in  Sub-process  5.  The  Risk 
Management  Plan  should  address  the  elements  of  Risk  Management  including  Risk  Identification, 

Risk  Analysis,  Risk  Assessment,  and  Risk  Handling.  Plans  for  a  Risk  Management  Board  and  Risk 
Reporting  should  be  defined.  Also  see  the  DAU  Publication  “Risk  Management  Guide  for  DoD 
Acquistion”  and  NAVAIRINST  5000.21,  NAVAIR  Program/Project  Risk  Management.  . 

c)  Technical  Review  Plan.  A  review  plan  shall  identify  any  significant  technical  reviews  required,  when 
they  will  occur,  and  the  purpose  of  the  review.  Typically  the  Review  Plan  is  not  a  stand-alone 
document  but  is  incorporated  in  the  SEP  (task  a)  above)  and  in  other  program  documentation.  The 
normal  sequence  of  reviews  for  a  typical  system  is:  System  Requirements  Review  (SRR);  System 
Functional  Review/Software  Specification  Review  (SFR/SSR);  Preliminary  Design  Review  (PDR); 
and  Critical  Design  Review/Test  Readiness  Review  (CDR/TRR).  The  nomenclature  and  acronyms  for 
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these  reviews  are  often  modified  for  specific  programs,  but  the  purpose  of  the  reviews  should  not 
change.  The  DoD  5000  series  provides  guidance  on  the  timing  of  major  reviews  relative  to  milestones. 
NAVAIR  Instruction  4355. 19B  describes  the  NAVAIR  Systems  Engineering  Technical  Review 
Process  to  be  used  for  technical  and  design  reviews.  When  preparing  a  technical  review  plan, 
coordination  is  required  to  ensure  that  the  appropriate  contractors  are  tasked  in  the  SOW  to  support  the 
reviews,  and  that,  if  reviews  are  tied  to  entry/exit  criteria  for  milestone  decisions,  it  is  reflected  in  the 
plan.  A  sample  of  an  event-based  schedule  of  reviews  is  contained  in  Appendix  A  of  the  DAU 
Systems  Engineering  Fundamentals. 

d)  Verification  Plans.  Verification,  as  well  as  Validation  (task  e)  below),  is  usually  accomplished  via 
some  form  of  testing.  The  relationships  of  the  various  test  plans  are  shown  in  Figure  4.2.1b 


Figure  4.2.1b  -  Test  Plan  relationships 

Verification  Plans  take  many  forms  depending  on  the  life-cycle  phase  and  program  content.  Sub¬ 
processes  30-32  require  Verification  Plans  that  are  often  very  informal  and  consist  only  of  a 
Verification  Matrix.  A  Verification  Matrix  shows  how  every  requirement  will  be  verified  such  as  by 
analysis,  modeling  and  simulation,  lab  test,  or  full-scale  test.  For  certain  critical  systems,  such  as 
digital  flight  control  systems,  a  separate  group  may  perform  verification  and  validation  tasks 
independent  of  the  developer.  These  efforts  will  be  defined  in  the  Independent  Verification  & 
Validation  (IV&V)  Plan,  DI-NDTI-80566. 

Sub-process  30,  Design  Solution  Verification,  is  usually  addressed  through  a  series  of  more  detailed 
verification  plans  or  Qualification  Test  Plans.  Qualification  Tests  are  usually  conducted  by  the 
contractor  in  a  laboratory  or  chamber  and  consist  of  tests  such  as  temperature/altitude,  shock, 
vibration,  and  EMI  for  “black  box”  type  systems  or  static  strength  or  fatigue  tests  for  mechanical  or 
structural  systems.  Plans  for  these  types  of  tests  are  tailored  for  the  environment  for  which  the  system 
is  being  designed  based  on  requirements  defined  in  the  system  specification.  These  tests  are  typically 
defined  in  the  contractor’s  SOW  and  the  verification  (qual)  test  plans  are  written  by  the  contractor  and 
approved  by  the  government. 
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Sub-process  31,  End  Product  Verification,  implies  a  formal  DT  (Developmental  Testing)  period, 
which  includes  both  testing  performed  by  the  contractor  or  developer  and  testing  performed  by  a 
government  or  integrated  test  team.  The  overarching  plan  for  testing  of  any  system  is  usually  the  Test 
and  Evaluation  Management  Plan  (TEMP).  Guidelines  for  TEMP  preparation  are  contained  in  the 
DoD  5000  series  documents.  TEMP  preparation  is  the  responsibility  of  the  Program  Manager  and 
requires  the  concurrence  of  all  key  parties  such  as  DOT&E,  COMOPTEVFOR,  N-912,  the  resource 
sponsor,  and  the  PEO.  Test  plans  for  specific  DT  tests  are  usually  developed  by  the  testing  activity 
(such  as  NAWCWD  or  NAWCAD)  and  are  prepared  in  their  format.  Contractor  test  plans  are  usually 
prepared  as  a  contract  deliverable  for  government  approval  prior  to  the  start  of  each  phase  of  testing 
such  as  EMI  testing.  Major  programs  usually  have  a  Test  and  Evaluation  Process  Working  Group 
(TEPWG),  which  has  the  responsibility  and  oversight  for  preparation  and  planning  of  all  major  DT 
events. 

e)  Validation  Plans.  Planning  for  validation  (OT  for  major  programs)  is  encompassed  in  the  TEMP.  A 
detailed  OT  Test  Plan  is  prepared  by  the  OT  Test  Activity  (e.g.,  VX-9)  and  approved  by 
COMOPTEVFOR. 

f)  Other  applicable  plans  as  called  for  in  the  agreement  or  by  enterprise  policies  and  procedures 

such  as  a  Configuration  Management  Plan,  Quality  Assurance  Plan,  Data  Management  Plan, 
Manufacturing  Plan,  Source  Selection  Plan,  and  Security  Management  Plan.  Sample  outlines  for  some 
of  these  plans  are  listed  below: 

Manufacturing  Plan  (see  DID  -DI-MISC-81180) 

(1)  Introduction  -  Background,  Manufacturing  Organization,  Management  System 

(2)  Manufacturing  Management  Program  -  Time  Phased  Schedule,  Manpower  Plan,  Industrial 

Facilities  Capacity  Assessment,  Risk  Assessment,  and  Capital  Investment  Commitment 

(3)  Manufacturing  Program  Planning  -  Producibility  Plan,  Make  or  Buy  Criteria,  Supplier 

Management,  Methods  and  Production  Flow,  Tooling  and  Special  Test  Equipment,  Productivity 

Improvement,  Industrial  Materials  Management 

(4)  Manufacturing  Management  Data 

(5)  Audits 

(6)  Labor  Relations 


QA  Plan  (see  ISO  9001) 

(1)  Quality  Management  System 

(2)  Management  Responsibility 

(3)  Resource  Management 

(4)  Product  Realization 

(5)  Measurement,  Analysis,  and  Improvement 

Parts  Management  Plan 

Use  MIL-  HDBK-512  as  guidance  and  a  source  of  additional  reference  material. 

Configuration  Management  Plan 

Use  MIL-HDBK-61  as  guidance.  NAVAIR  INST  4130.1C  provides  details  on  the  CM  process. 
Requirements  for  a  contractor’s  Configuration  Management  Plan  are  found  in  DI-CMAN-80858B. 

Source  Selection  Plan  (SSP) 


Refer  to  Appendix  D  for  various  types  of  plans  that  may  be  considered  for  development  by  this  sub-process. 
Outputs 
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All  outputs  should  be  archived  (SP  12) 

•  System  Engineering  Plan  (SEP)  and/or  Software  Development  Plan  (SDP)  (SP  9,  10,  11,  22,  24,  30) 

•  Test  and  Evaluation  Management  Plan  (TEMP)  (SP  2,  1 1 ,  30,  3 1 ,  33) 

•  Risk  Management  Plan  (SP  24) 

•  Computer  Resources  Life  Cycle  Management  Plan  (CRLCMP)  (SP  9) 

•  Configuration  Management  Plan  (SP  5,  9) 

•  Quality  Assurance  (QA)  Program  Plan  (SP  20) 

•  Manufacturing  Plan  (SP  20) 

•  Data  and  Document  Management  Plan  (SP  5,13) 

•  Security  Management  Plan  (SP  ALL) 

•  Verification  Plan  (including  the  Verification  Compliance  Requirement  Matrix  (VCRM))  (SP  25,  30,  31) 

•  Validation  Plan  (to  include  what  NAVAIR  calls  Operational  Test  Plan  and  Developmental  Test  Plan)  (SP 
11,25,26,  27,  28,  29,33) 

•  Independent  Verification  and  Validation  (IV&V)  Plan  (SP  30) 

(for  early  development  testing  typically  for  software,  done  by  a  3rd  party) 

•  Source  Selection  Plan  (SSP)  (SP  2) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

All  technical  plans  identified,  written,  and  approved. 

Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Sub-process  10:  Progress  Against  Requirements 
Sub-process  ll:Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Implementation  Process 

Sub-process  20:  Implementation 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  24:  Risk  Analysis 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 
Sub-process  26:  Acquirer  Requirements  Validation 
Sub-process  27:  Other  Stakeholder  Requirements  Validation 
Sub-process  28:  System  Technical  Requirements  Validation 
Sub-process  29:  Logical  Solution  Representations  Validation 
System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 
Sub-process  31:  End  Product  Verification 
End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Agents 

Acquirer,  Systems  Engineering,  Program  Manager,  Test  Engineers,  COMOPTEVFOR,  Contractors 
Tools 

Planning  and  scheduling  tools  (ex.  Microsoft  Project) 

Automated  Systems  Engineering  tools  (ex.  CORE™,  SLATE™) 
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References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals  (Chapter  1 6) 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

DI-MGMT-81024,  Systems  Engineering  Management  Plan  (SEMP) 

DI-NDTI-80566 

DI-MISC-81180 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Project  Planning  process  areas 
Earned  Value  Management  System  (EVMS)  Industry  Standards  (EIA-748),  1998 

DI-CMAN-80858B,  Contractor’s  Configuration  Management  Plan 
ISO/IEC  12207 
IEEE  1220 

DAU:  Risk  Management  Guide  for  DoD  Acquisition 

ISO  9001 

MIL-  HDBK-512A 

MIL-HDBK-61A 


NAVAIR  Specific: 

•  NAVAIR  INST  4130.1C  Configuration  Management  Policy 

•  NAVAIR  INST  4355.19B  Systems  Engineering  Technical  Review  Process 

•  NAVAIR  INST  5000.21  Program/Project  Risk  Management 

•  APMSE  Quick  Reference  Guide 


Metrics  and  Measures 

Plans  completed  and  released  on  time. 

The  expected  outcomes  for  the  tasks  related  to  developing  these  plans  are  provided  in  Appendix  C.  The 
outcomes  associated  with  completing  this  sub-process  provide  guidance  for  preparing  work  directives  and 
completing  other  applicable  project  processes  for  engineering  a  system. 

Any  plan  created  should  include  the  scope,  tasks,  methods,  tools,  metrics,  risks,  and  resources  as  applicable  to 
fulfill  the  purpose  of  the  plan. 


NOTE  -  Appendix  D  of  this  Guide  contains  a  listing  of  typical  planning  documents.  Some  projects 
require  either  more  or  significantly  less  documentation.  These  planning  documents  can  be  tailored  as  to 
the  level  and  formality  of  planning  to  suit  project  complexity  and  uncertainty. 


Sub-process  8  -  Work  Directives 

The  developer  shall  create  work  directives  that  implement  the  planned  technical  effort. 


Preceding  Process 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  4:  Process  Implementation  Strategy 
Sub-process  5:  Technical  Effort  Definition 
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Sub-process  6:  Schedule  and  Organization 
Requirements  Definition 

Sub-process  16:  System  Technical  Requirements 

Inputs 

•  Process  Implementation  Strategy  (SP  4) 

•  Life  Cycle  Phase  Chart  (SP  4) 

•  Total  Life  Cycle  Cost  Objectives  (SP  5) 

•  Life-cycle  phase  exit  criteria  (SP  4) 

•  Organizational  Structure  (SP  6) 

•  Integrated  Master  Schedule  (SP  6) 

•  Inputs  to  Earned  Value  Management  System  (EVMS)  (SP  5) 

•  Cost,  schedule,  and  performance  constraints  (SP  2) 

•  System  Technical  Requirements  (SP  16) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Develop  individual  project  team  or  organization  work  packages  that  describe  the  work  to  be 
done,  resource  sources,  schedules,  budget,  and  reporting  requirements. 

Statement  of  Work  (SOW).  The  Statement  of  Work  (SOW)  is  a  portion  of  a  contract  which 
establishes  and  defines  all  non-specifications  requirements  for  contractors’  efforts  either  directly  or 
with  the  use  of  specific  cited  documents.  See  MIL-STD-245D. 

Statement  of  Objectives  (SOO).  The  Statement  of  Objectives  (SOO)  is  a  portion  of  a  contract  which 
establishes  a  broad  description  of  the  governments’  required  performance  objectives. 

Team  Work  Plan  (TWP).  The  Team  Work  Plan  (TWP)  addresses  labor  by  category,  material,  travel, 
flight  costs,  expendables,  range  requirements  and  laboratory  requirements.  The  TWP  might  include:  a 
program  summary,  cancellations,  references,  and/or  enclosures;  technical  instructions;  schedule; 
reports  and  documentation  to  be  provided;  future  planning  information;  contractual  authority;  source 
and  disposition  of  equipment;  and  security  classifications. 

b)  Generate  work  authorizations  for  the  team  or  organization  that  provide  approval  for  applicable 
teams  or  organizations  to  complete  their  work  package  requirements  and  to  release  applicable 
resources. 

NAVAIR  Specific: 

Team  Assignment  Agreement  (TAA).  NAVAIR  has  instituted  the  Team  Assignment  Agreement 
(via  NAVAIRINST  5400.154  dated  15  August  2000)  as  the  vehicle  to  establish  the  process  and 
procedures  within  NAVAIR  for  the  assignment  of  its  personnel  to  Teams.  It  documents  the  method  to 
be  used  to  describe  the  work  to  be  done,  resources,  schedules,  funding,  and  reporting  requirements  for 
competency  support.  The  program  offices  may  use  a  different  mechanism  for  setting  their  internal 
resource  requirements. 

The  final  product  is  the  signed  Team  Assignment  Agreement  (TAA)  that  meets  both  the  program  and 
competency  requirements.  The  TAA  should  address  the  following:  tasks,  functions,  products,  and/or 
services  to  be  provided;  funding  summary;  availability/duration  of  resources;  authority/empowerment 
level;  training  requirements  and  agreements;  collocation  requirements;  performance  evaluation  inputs 
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required;  administrative  functions  delegated  to  Team  leadership;  and  the  issue  resolution  process  to  be 
employed. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Team  Work  Plan  (TWP)  (SP  2,  15,  30) 

•  Statement  of  Objectives  (SOO)  (SP  2,  15,  30) 

•  Statement  of  Work  (SOW)  (SP  2,15,30) 

NAVAIR  Specific: 

•  Team  Assignment  Agreement  (TAA)  (SP  1) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

(TAA  Signed,  WBS  defined) 

Next  Processes 
Supply  Process 

Sub-process  1 :  Product  Supply 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  15:  Other  Stakeholder  Requirements 
System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 

Agents 

Acquirer:  PEO/PM,  IPT,  Systems  Engineering,  Logistics 

Tools 

WBS 

TAA  Form 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Earned  Value  Management  System  (EVMS)  Industry  Standards  (EIA-748),  1998 

MIL-HDBK-881  Work  Breakdown  Structure;  2  January  1998 

MIL-STD-245D 


NAVAIR  Specific: 

•  NAVAIR  TAA  Instruction  (NAVAIRINST  5400.154) 

Metrics  and  Measures 

Risk  Cube 

EVMS 

WBS 

Capability  Maturity 
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The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  the  means  to  implement  the  planned  technical  effort. 

4.2.2  Assessment  Process 

The  Assessment  Process  is  used  to:  (1)  determine  progress  of  the  technical  effort  against  both  plans  and 
requirements;  (2)  review  progress  during  technical  reviews;  and  (3)  support  control  for  the  engineering  of  a 
system.  The  product  and  process  metrics  selected  for  assessing  progress  should  provide  information  for  risk 
aversion,  meaningful  financial  and  non-fmancial  performance,  and  support  of  project  management. 


NOTE  -  When  variations  are  sufficiently  significant  or  cannot  be  corrected  by  re-accomplishment  of  the 
process  tasks  that  generated  the  outcome  data,  the  Planning  Process  is  re-initiated  in  order  to  implement 
appropriate  corrective  actions. 


The  three  sub-processes  associated  with  the  Assessment  Process  are  shown  in  Figure  4.2.2a. 


Assessment 

Process 

Requirements 


Sub-process  9  -  Progress  Against  Plans  and  Schedules 


•Sub-process  10  -  Progress  Against  Requirements 


Sub-process  11  -  Technical  Reviews 


Figure  4.2.2a  -  Assessment  Process/Sub-processes 

Inputs  to  the  Assessment  Process  are  in  the  form  of  technical  plans,  stakeholder  requirements,  and  engineering 
outcomes  from  other  processes.  The  relationships  of  the  Assessment  Process/Sub-processes  are  shown  in 
Figure  4.2.2b. 

These  sub-processes  use  metrics  produced  by  an  EVM  system  (see  Sub-process  5)  to  track  the  progress  of  the 
processes.  Product  technical  requirements  essential  to  the  system  being  acquired  are  also  tracked.  Sub-process 
9  uses  metrics  to  track  the  progress  against  the  program  plans  and  schedules  used  to  manage  the  program,  while 
Sub-process  10  tracks  the  progress  in  meeting  product-related  technical  requirements.  Sub-process  11 
provides  a  status  of  design  maturity  and  requirement  satisfaction,  identifies  risks  and  issues  to  be  resolved  and 
determines  whether  the  system  is  ready  for  the  next  engineering  phase.  Cost,  schedule  and  performance 
variances  reflected  in  the  metrics  are  fed  into  a  risk  management  system  (see  Sub-process  24),  which  produces 
a  risk  management  system  with  risk  mitigations  identified,  the  effect  of  which  can  be  observed  and  adjusted.  A 
program,  which  does  not  employ  a  closed  loop  to  feed  EVM  system  variances  into  the  risk  management  system 
cannot  be  effective  in  making  positive  changes  in  the  management  of  the  system. 


Figure  4.2.2b  -  Assessment  Process/Sub-processes  relationships 
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Sub-process  9  -  Progress  Against  Plans  and  Schedules 

The  developer  shall  assess  the  progress  of  the  program  effort  against  applicable  plans, 
schedules,  and  budgets. 


Preceding  Process 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Systems  Analysis  Process 

Sub-process  23:  Trade-off  Analysis 

Inputs 

•  Technical  Performance  Measurements  (TPM)  (SP  5) 

•  Work  Breakdown  Structure  (WBS)  (SP  5) 

•  Inputs  to  Earned  Value  Management  System  (EVMS)  (SP  5) 

•  Program  metrics  (SP  5) 

•  Process  metrics  (SP  5) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Systems  Engineering  Plan  (SEP)  or  Software  Development  Plan  (SDP)  (SP  7) 

•  Computer  Resources  Life  Cycle  Management  Plan  (CRLCMP)  (SP  7) 

•  Configuration  Management  Plan  (SP  7) 

•  Trade-off  Analysis  Technical  Report  (SP  23) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Tasks 

The  developer  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process  using  an  Earned  Value 
Management  System  (EVMS)  as  described  in  Sub-process  5.  Tasks  to  consider  include  the  following: 

a)  List  the  appropriate  events  such  as  system  specification,  design  reviews,  tasks,  and  process 
metrics,  including  capability  maturity,  for  monitoring  progress  against  plans  and  schedules. 

b)  Collect  and  analyze  identified  process  metrics  data  and  results  from  completion  of  planned  and 
scheduled  tasks  and  events,  which  will  be  used  to  conduct  trend  analyses.  Assess  the  program’s 
schedule  performance  status  by  examining  data  produced  by  an  EVMS.  Compare  the  actual  or 
forecast  dates  and  durations  to  the  targeted  dates  and  durations.  Collect  the  number  of  actual  hours 
worked  from  the  accounting  system. 

c)  Compare  process  metrics  data  against  plans  and  schedule  using  trend  analysis  to  determine 
technical  areas  requiring  management  or  team  attention.  Compare  the  actual  or  forecast  hours  to 
target  hours.  Continually  identify  and  manage  critical  path  activities. 

d)  Determine  risk  and  identify  need  to  correct  variances,  make  changes  to  plan  and  schedule,  and 
redirect  work  because  of  risk. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  List  of  appropriate  events,  tasks,  and  process  metrics  (SP  9) 

•  Process  metrics  data  (SP  9) 

•  Program  metrics  data  (SP  9) 
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•  Plans  and  schedules  trend  analysis  (SP  3,  9,  11,  12,  23,  24) 

•  Cost  Performance  Report  (CPR  or  C/SSR)  (SP  12) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Next  Processes 
Acquisition  Process 

Sub-process  3 :  Supplier  Performance 
Assessment  Process 

Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Systems  Analysis  Process 

Sub-process  23:  Trade-off  Analysis 
Sub-process  24:  Risk  Analysis 

Agents 

Acquirer 

Stakeholder 

Program  Management 

Systems  Engineering 

Logistics 

Cost 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

Schedule  software  with  Insight  (ex.  MS  Project,  Open  Plan  Professional,  Primavera,  etc) 

Completion  Date  Histogram 
Logic  Diagrams 
Gantt  Bar  Charts 
Milestone  Charts 
Resource/Hour  Usage  Charts 

Earliest,  Expected  and  Latest  Completion  Dates  and  Durations 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Earned  Value  Management  System  (EVMS)  Industry  Standards  (EIA-748),  1998 

DAU  Program  Manager’s  Tool  Kit,  2004 

DRAFT  MIL-STD-499B  Systems  Engineering 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Project  Monitoring  and  Control  process 
areas 


NAVAIR  Specific: 

•  NAVAIR  Acquisition  Guide 

•  NAVAIRINST  4355.19B,  Systems  Engineering  Technical  Review  (SETR)  Process 
Metrics  and  Measures 
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Percent  EVMS  that  is  not  level  of  effort 
Accuracy  of  trend  analysis 

Amount  of  time  between  the  closing  of  a  reporting  period  and  the  reporting  of  a  metric 
Number  of  team  members  that  have  access  to  their  appropriate  metrics 
IPT  member  satisfaction  with  the  metrics 
Provided  EVMS  metrics  used 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  status  information  to  enable  efficient  use  of  resources,  evaluation  of 
progress  against  plan,  identification  of  variances  of  cost  and  schedule  from  planned  project  management 
baselines,  and  early  identification  and  resolution  of  productivity  problems. 


NOTE  -  Process  metrics  are  identified  and  used  to  assess  the  means  of  attaining  stakeholder  satisfaction. 
Process  metrics  include  earned  value  (cost/schedule  measure),  amount  of  waste,  number  of  engineering 
changes,  percentage  of  drawings  completed,  number  of  drawing  errors,  percentage  of  lines  of  code 
completed,  rework  percentage,  idle  time  (e.g.,  work  in  progress),  change  rate,  and  turnover  in  personnel.  The 
criteria  for  process  metric  selection  are  based  on  how  well  enhancement  in  project  performance  correlates 
with  improvement  in  potential  customer  satisfaction. 


Sub-process  10  -  Progress  Against  Requirements 

The  developer  shall  assess  the  progress  of  system  development  by  comparing  currently 
defined  system  characteristics  against  requirements. 


Preceding  Process 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Assessment  Process 

Sub-process  11:  Technical  Reviews 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Sub-process  16:  System  Technical  Requirements 
System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 
Sub-process  31:  End  Product  Verification 

Inputs 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD)  (SP 
14) 

•  Systems  Engineering  Plan  (SEP)  or  Software  Development  Plan  (SDP)  (SP  7) 

•  Technical  Performance  Measurements  (TPM)  (SP  5) 

•  Work  Breakdown  Structure  (WBS)  (SP  5) 

•  Key  Performance  Parameters  (KPP)  (SP  16) 

•  Product  metrics  (SP  5) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Technical  review  report  (SP  11) 

•  Design  solution  deficiency  and  discrepancy  reports  (SP  30) 

•  End  product  deficiency  and  discrepancy  reports  (SP  31) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 
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Develop  enterprise  architecture  data  that  includes,  but  not  limit  to,  program  goals,  thresholds,  objectives,  user 
requirements,  cost,  schelude,  and  performance  parameters.  The  architecture  data  should  be  in  a  Core 
Architectural  Data  Model  (CADM)-base  repository  or  CADM  compliance  repository,  which  can  be  used  to 
assess  system  capabilities  and  shortfalls  and  to  include  the  associated  costs  and  schedule  for  providing  those 
capabilities. 

Tasks 

The  developer  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Identify  product  metrics,  and  their  expected  values,  that  will  affect  the  quality  of  the  product 
and  provide  information  of  the  progress  toward  satisfying  acquirer  and  other  stakeholder 
requirements,  as  well  as  derived  requirements.  Integrated  Product  team  (IPT)  leaders  or  functional 
managers  identify  Key  Performance  Parameters  (KPPs)  and  Technical  Performance  Measures  (TPMs) 
to  be  tracked.  (See  Sub-process  5.)  TPMs  are  added  or  deleted,  or  parameters  adjusted  as  the 
program  progresses  to  ensure  that  an  appropriate  set  of  key  performance  requirements  is  being 
monitored  (and  managed). 

b)  Collect  and  analyze  product  metrics  data.  This  is  typically  done  by  the  IPT  to  conduct  trend 
analysis.  Examples  might  include,  power,  sensitivity,  vibration,  fuel  consumption,  weight,  balance  and 
software  function  points.  A  technical  compliance  matrix  is  used  to  compare  actual  progress  with  the 
requirements  baseline  (or  plan). 

c)  Record  rationale  for  decisions  and  assumptions  made  with  respect  to  collected  data. 

d)  Compare  results  against  requirements  to  determine  degree  of  technical  requirement  satisfaction, 
progress  toward  maturity  of  the  system  (or  portion  thereof)  being  engineered,  and  variations  and 
variances  from  requirements. 

e)  Identify  deficiencies  and  discrepancies  to  specifications  and  configuration  baselines.  This  is 
important  to  Sub-process  5,  Sub-process  7,  and  Sub-process  14  to  consider  revisions  to  technical 
approaches,  requirements  and/or  plans  in  the  event  that  it  appears  that  one  or  more  requirements  will 
not  be  able  to  be  met  as  presently  defined.  It  may  be  necessary  to  change  a  technical  approach  or 
revise  a  requirement  if  the  requirements  cannot  be  met. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Requirement  trend  analysis  (requirement  satisfaction,  system  maturity,  technical  compliance  matrix)  (SP  3, 
11,23,24) 

•  Deficiencies  and  discrepancies  (SP  11,  19) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Next  Processes 
Acquisition  Process 

Sub-process  3 :  Supplier  Performance 
Assessment  Process 

Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
Systems  Analysis  Process 

Sub-process  23:  Trade-off  Analysis 


47 


Sub-process  10 


Sub-process  24:  Risk  Analysis 


Agents 

Program  Management 
Systems  Engineering 
Logistics 
Cost 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

Schedule  software  w/Insight  (ex.  MS  Project,  Open  Plan  Professional,  Primavera,  etc) 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Earned  Value  Management  System  (EVMS)  Industry  Standards  (EIA-748),  1998 

DAU  Program  Manager’s  Tool  Kit,  2004 
DRAFT  MIL-STD-499B  Systems  Engineering 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Project  Monitoring  and  Control  process 
areas 


NAVAIR  Specific: 

•  NAVAIR  Acquisition  Guide 

•  NAVAIRINST  4355.19B  Systems  Engineering  Technical  Review  (SETR)  Process 
Metrics  and  Measures 

Percent  requirements  (appropriate  to  the  level  of  development)  that  have  been  analyzed,  and  percent 
deficiencies  and  discrepancies  identified  and  reported  to  the  appropriate  agents. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  Representative  outcomes 
associated  with  completing  this  sub-process  provide:  (1)  an  evaluation  of  the  progress  toward  meeting 
requirements  pertaining  to  the  system  being  engineered  or  reengineered;  (2)  status  information  to  enable 
efficient  use  of  resources;  (3)  evaluation  and  tracking  of  system  quality  and  technology;  (4)  faster  response  time 
to  inquiries  from  acquirer  or  other  stakeholders;  (5)  identification  of  variances  from  planned  improvements  in 
critical  technical  parameters  as  the  design  evolves;  (6)  early  identification  and  resolution  of  system  related 
problems;  and  (7)  tracking  trade-off  analysis  and  analysis  of  alternative  recommendations,  effectiveness 
analysis  results,  verification  outcomes,  and  validation  results. 


NOTE  -  Product  metrics  are  used  to  measure  stakeholder  satisfaction,  deliver  an  ever- improving  value  to 
the  acquirers  of  system  end  products,  and  be  indicative  that  the  design  process  is  continuing  toward  an 
acceptable  solution.  An  example  of  an  input  product  metric  is  the  quality  of  materials  and  skills  of 
assigned  project  personnel.  An  example  of  an  output  metric  is  a  Technical  Performance  Measure  (TPM). 


Sub-process  11  -  Technical  Reviews 

The  developer  shall  conduct  technical  reviews  of  progress  and  accomplishments  in 
accordance  with  appropriate  technical  plans. 


Preceding  Process 
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Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  6:  Schedule  and  Organization 
Sub-process  7:  Technical  Plans 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 
Sub-process  31:  End  Product  Verification 

Inputs 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD)  (SP 
14) 

•  Testing  metrics  (SP  5) 

•  Technical  Performance  Measurements  (TPM)  (SP  5) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Validation  Plan  (SP  7) 

•  Systems  Engineering  Plan  (SEP)  or  Software  Development  Plan  (SDP)  (SP  7) 

•  Test  &  Evaluation  Master  Plan  (TEMP)  (SP  7) 

•  Plans  and  schedules  trend  analysis  (SP  9) 

•  Requirement  trend  analysis  (SP  10) 

•  Deficiencies  and  discrepancies  (SP  1 0) 

•  Systems  Requirements  Document  (SP  1 6) 

•  System  technical  requirements  (SP  1 6) 

•  Specified  requirements  (SP  19) 

•  Design  solution  deficiency  and  discrepancy  reports  (SP  30) 

•  End  product  deficiency  and  discrepancy  reports  (SP  31) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Tasks 

Technical  reviews  are  conducted  to  ensure  that  the  product  being  developed  meets  the  requirements  for  the 
appropriate  anticipated  level  of  maturity.  Each  review  must  have  defined  entry  and  exit  criteria  tied  to  the 
required  level  of  design  maturity  and  applied  across  all  requirements  and  technical  disciplines. 

The  developer  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process.  Within  NAVAIR, 
NAVAIRINST  4355. 19B  establishes  the  policies  and  responsibilities  for  conducting  technical  reviews,  a 
detailed  description  of  the  types  of  reviews,  and  the  duties  of  participants. 

Tasks  to  consider  include  the  following: 

a)  Identify  the  review  objectives  and  requirements  cited  in  the  Systems  Engineering  Plan  (SEP); 
enterprise  policies  and  procedures;  and  agreement,  as  applicable. 

b)  Verify  completion  of  the  technical  review  entry  requirements. 
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1)  Identify  the  anticipated  completion  at  that  stage  of  maturity  (TPMs,  drawings)  evaluated  against 
the  anticipated  status/requirements. 

2)  Confirm  that  necessary  reviews,  inspections,  tests,  processes,  deliveries,  and  coding  were 
completed  properly  as  specified/required. 

c)  Establish  the  technical  review  board,  agenda,  and  speakers. 

d)  Prepare  the  appropriate  materials  to  include  in  the  read-ahead  technical  review  package  and 
presentation  package. 

e)  Facilitate  and  support  identification  and  resolution  of  emerging  issues  prior  to  the  review. 

f)  Conduct  the  technical  review  using  the  guidance  of  the  Design  Review  Handbook  according  to 
the  SEP,  identifying  and  documenting  action  items  required  to  meet  the  review  objectives. 

1)  Evaluate  the  design  for  compliance  with  known  technical  requirements. 

2)  Verify  interfaces  compatibility. 

3)  Determine  what  issues  remain  to  be  resolved. 

4)  Verify  that  the  emerging  design  is  ready  to  enter  the  next  stage  of  development. 

5)  Verify  that  the  product  is  testable,  manufacturable,  usable,  safe  and  reliable. 

6)  Verify  that  the  product  exhibits  the  characteristics  necessary  to  prove  effective  and  suitable  during 
operational  evaluation  throughout  the  development  phase. 

7)  Challenge  the  design  and  related  processes  for  optimization. 

8)  Communicate  requirements,  design  concepts  and  descriptions  to  other  departments. 

g)  Close  out  the  review  after  (1)  minutes  have  been  prepared,  approved,  and  distributed;  (2)  action 
items  have  been  resolved;  and  (3)  the  review  has  been  signed-off  by  the  director.  Prepare  the 
Technical  Review  Report. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Technical  Review  Report  (TRR)  (SP  10) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Next  Processes 
Assessment  Process 

Sub-process  10:  Progress  Against  Requirements 
Control  Process 

Sub-process  12:  Outcomes  Management 

Agents 

Acquirer 

Stakeholders 

Program  Management 

Systems  Engineering 

Logistics 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 
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References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
MIL-STD-1521 

DRAFT  MIL-STD-499B 

DoD  4245.7-M  Transition  from  Development  to  Production  Chapter  3 
NAVSO  P-6071  Best  Practices  Section  4.0 

NAVAIR  Specific: 

•  NAVAIRINST  4355.19B  Systems  Engineering  Technical  Review  Process 

•  NAVAIR  Design  Review  Handbook  (AIR  4.1) 

Metrics  and  Measures 

Minutes  and  action  items  completed  and  accepted  by  the  appropriate  agent 

Functional  Allocation 

Performance 

Cost,  Schedule,  Weight 

Risk 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completion  of  this  sub-process  (1)  help  ensure  that  all  event-based  plan  criteria  have  been  met,  (2)  provide 
ongoing  status  of  design  maturity  and  how  well  the  concepts  satisfy  requirements,  (3)  provide  traceability  of 
requirements  and  validity  of  assumptions  and  decision  rationale,  (4)  provide  identification  of  issues  to  be 
resolved  and  those  issues  not  determined  during  the  development  effort,  and  (5)  highlight  related  risks,  needed 
resources,  and  preparation  for  conducting  the  next  engineering  life-cycle-phase  development  effort. 

4.2.3  Control  Process 

The  Control  Process  is  used  to:  (1)  manage  the  conduct  and  outcomes  of  the  Acquisition  and  Supply  Processes, 
System  Design  Processes,  Planning  and  Assessment  Processes,  Product  Realization  Processes,  and  Technical 
Evaluation  Processes;  (2)  monitor  variation  from  the  plan  and  anomalies  relative  to  requirements;  (3)  distribute 
required  and  requested  information;  and  (4)  ensure  necessary  communications.  This  process  supports 
satisfaction  of  the  agreement  and  assurance  that  variations  and  anomalies  are  corrected  by  repeating  appropriate 
tasks. 

The  two  sub-processes  associated  with  the  Control  Process  are  shown  in  Figure  4.2.3. 


‘Sub-process  12  -  Outcomes  Management 

.Sub-process  13  -  Information  Dissemination 


Figure  4.2.3  -  Control  Process/Sub-processes 

Inputs  to  the  Control  Process  are  in  the  form  of  outcomes  from  other  processes  plus  project  and  enterprise 
information  affecting  the  engineering  of  a  system. 


Control 

Process 

Requirements 
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Sub-process  12  -  Outcomes  Management 

The  developer  shall  manage  the  outcomes  of  the  technical  effort. 


Preceding  Processes 

All  other  Systems  Engineering  Processes  (Sub-processes  1-11,  13-33) 

Inputs 

Below  is  a  generalized  list  of  information  that  should  be  included  in  the  Enterprise  Data  Repository.  This  is  not 
an  all-inclusive  list.  It  should  include  all  outputs  of  all  Systems  Engineering  Processes  (Sub-processes  1 
through  33)  as  appropriate,  even  source  documentation  for  creating  items  in  the  below  list  should  be  included 
for  historical  records. 

•  Mission  Areas  (Navy  Mission  Essential  Task  List  (NMETL),  Mission  Capability  Packages  (MCPs),  Joint 
Task  Lists  (JTLs),  etc.) 

•  Solicitations 

•  Proposals 

•  Signed  agreements 

•  Program  plans 

•  Technical  plans 

•  Changes 

•  Stakeholder  information  (e.g.,  doctrine,  organization,  training,  material,  leadership  and  education,  people, 
and  facilities  (DOTMLPF)) 

•  Reference  documents 

•  Policies,  methods,  and  procedures 

•  Technical  Data  Packages 

•  Metrics 

•  Cost  objectives/information 

•  Work  Breakdown  Structure 

•  Schedules 

•  Life  Cycle  Support  Plans 

•  Program  Operating  Guides  (NAVAIR  unique) 

•  Analyses 

•  Reports 

•  Technical  presentations 

•  Requirements 

•  Traceability  matrix 

•  Trade  studies 

•  Functional  and  physical  baselines 

•  Certifications 

•  Specifications 

•  Systems  Engineering  Plan 

•  Deficiencies  and  discrepancies 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Tasks 

Outcomes  management  provides  for  the  capture  and  management  of  data  from  the  management  and  technical 
effort  for  the  program.  This  information  is  used  to  redirect  the  work  effort  to  overcome  obstacles,  to  respond  to 
changing  circumstances,  or  to  correct  variances.  An  Enterprise  Data  Repository  that  was  established  in  Sub¬ 
process  5  is  used  to  preserve  all  the  program’s  pertinent  information  that  is  needed  by  any  and  all  of  the 
program  stakeholders. 
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The  developer  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Capture  the  outcomes,  descriptions  of  methods  and  tools  used,  decisions  and  assumptions, 
lessons  learned,  and  other  data  that  allow  for  tracking  requirements. 

An  enterprise  data  repository  should  be  used  to  store  all  information  (doctrine,  organization,  training, 
material,  leadership  and  education,  people,  and  facilities  (DOTMLPF))  and  the  engineering  decisions 
used  and  generated  describing  the  current  state  of  the  system.  The  enterprise  process  model  and 
database  should  implement  a  traceability  matrix  that  maps  requirements  to  missions,  to  task  and 
operational  activivties,  to  system  functions,  and  then  on  to  systems.  The  result  of  this  knowledge  trace 
should  provide  a  clear  picture  of  the  enterprise  information  capabilities  and  shortfalls,  including  the 
inherent  associated  costs  for  providing  those  capabilities. 

The  database  should  be  a  Core  Architecture  Data  Model  (CADM)  or  CADM  compiliance  repository. 

It  should  be  shareable  (collaborative  enviroment)  so  that  team  members  have  access  to  the 
data/information  needed  in  a  native  environment,  to  ensure  persistent  and  correct  data.  The  integrated 
and  federated  database  must  be  accurate,  collaborative,  extensible,  interactive,  scalable,  web  enabled, 
unambiguous,  secure,  survivable,  easily  accessible  by  authorized  users,  and  complete.  The  project 
should  regularly  back  up  the  database  using  appropriate  media  to  enable  recovery  from  disaster,  failure 
of  equipment  or  media,  or  accidental  deletion  of  data. 

The  following  is  usually  recorded  in  the  information  database:  (1)  the  outputs  of  the  technical 
processes,  including  results  from  assessments;  descriptions  of  methods,  tools,  and  metrics  used;  and 
recommendations,  decisions,  assumptions,  and  impact  of  work  and  decisions;  (2)  lessons  learned;  (3) 
deviation  from  plan;  (4)  anomalies  and  out-of-tolerances  relative  to  requirements;  (5)  other  data  that 
allow  for  tracking  requirements;  and  (6)  doctrine,  organization,  training,  material,  leadership  and 
education,  people,  and  facilities  (DOTMLPF). 

The  front  end  of  the  enterprise  database  should  be  the  architecture  framework  and  framework/ 
environment  upon  which  a  systems  engineering  process  will  ride.  The  products  of  a  framework, 
including  activity  diagrams,  state  transition,  rules,  event  trace,  etc.,  are  all  the  front  end  equivalents  of 
the  functional  flow  and  data  flow  models  of  the  systems  engineering  process  extended  to  the 
operational  view  of  the  enterprise.  Further,  the  system  interface  diagrams,  physical  diagrams, 
information  exchange  requirements,  etc.,  are  all  instances  of  the  systems  design  side  of  the  systems 
engineering  process.  These  concepts  further  extend  into  the  areas  of  performance,  schedule,  risk, 
budget,  and  modeling  and  simulation.  Capturing  decisions,  assessments  and  rationale  in  architecture 
products  is  important  for  a  number  of  reasons:  it  gives  a  context  to  requirements  and  specifications;  it 
is  useful  when  assessing  the  impact  of  downstream  requirements  changes;  it  captures  hidden 
assumptions;  and  it  acts  as  a  requirement  filter.  Capture  of  rationale  with  each  requirement  often  helps 
uncover  the  actual  need  that  the  statement  of  the  requirement  intended  to  identify. 

b)  Perform  configuration  management  in  accordance  with  the  Configuration  Management  Plan.  In 
doing  this  activity,  the  following  tasks  should  be  considered  in  accordance  with  the  Configuration 
Management  Plan  (Sub-process  7). 

1)  Identify  documents  comprising  the  configuration  baselines  for  the  system  and  lower  level  items, 
and  put  them  under  configuration  control. 

2)  Control  of  all  proposed  changes  to  the  established  configuration  documentation. 

3)  Maintain  and  report  information  as  to  the  disposition  and  implementation  of  change  actions  and  as 
to  current  configuration  status  to  appropriate  stakeholders. 

4)  Perform  audits,  including  verification  that  the  system  elements  conform  to  the  current  approved 
specified  requirements  and  documentation. 
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c)  Perform  change  management  in  accordance  with  the  change  management  plan.  In  doing  this  activity, 
the  following  tasks  should  be  considered  in  accordance  with  the  Change  Management  Plan  (Sub¬ 
process  7) 

1)  Establish  formal  procedures  for  the  initiation,  assessment,  review,  approval,  and  disposition  of 
changes  to  agreements  and  approved  project  requirement  baselines,  configuration  baselines,  plans, 
and  work  directives. 

2)  Identify  and  track  proposed  and  directed  changes  to  agreements  and  approved  project 
requirements,  configuration  baselines,  plans,  work  directives,  or  any  other  action  or  activity  that 
would  affect  the  outcome  of  the  project. 

3)  Analyze  each  change  to  determine  the  impact  to  the  system,  the  system  product,  and  the  remaining 
requirements. 

4)  Analyze  the  cost,  schedule,  performance  and  risks  associated  with  making  a  proposed  or  directed 
change  within  schedule  and  resource  availability. 

5)  Maintain  and  control  traceability  of  changes  including  sources  of  the  change,  processing  methods, 
and  approvals  in  accordance  with  the  Change  Management  Plan. 

6)  Disseminate  the  approved  change  information/data  for  implementation. 

7)  Update  the  agreement  appropriately  in  all  cases  where  a  negotiated  and  approved  change  proposal 
affects  the  conditions  of  the  agreement. 

d)  Perform  interface  management  in  accordance  with  the  interface  management  plan.  In  doing  this 
activity,  the  following  tasks  should  be  considered  in  accordance  with  the  interface  management  plan 

(Sub-process  7) 

1)  Identify  internal  and  external  physical  and  functional  interfaces  that  exist  between  products, 
functions,  and  tasks  that  are  defined  from  other  process  activities  (e.g.,  agreement,  specification, 
system  product  tree,  WBS,  building  block  hierarchy). 

2)  Establish  interface  management  responsibilities  for  those  interfaces  that  are  part  of  the  agreement 
boundaries. 

3)  Maintain  and  control  identified  internal  and  external  physical  and  functional  interfaces  including 
completion  of  interface  definitions,  assessments  of  compatibility,  changes,  and  coordination  and 
approvals  with  appropriate  stakeholders. 

4)  Prepare  and  maintain  appropriate  physical  and  functional  interface  specifications  or  interface 
control  documents/drawings  to  describe  and  control  interfaces  external  to  the  system  products, 
interfaces  between  system  elements,  and  interfaces  among  configuration  management  items  in 
accordance  with  the  Interface  Management  Plan  and  project  directives  or  procedures. 

5)  Establish  and  implement  formal  change  procedures  for  interface  evolution. 

6)  Disseminate  the  needed  interface  information/data  for  implementation  and  control. 

e)  Perform  risk  management  in  accordance  with  the  Risk  Management  Plan.  Risk  analysis  is 
performed  in  Sub-process  24  but  is  managed  in  this  sub-process.  Both  are  done  in  accordance  with 
the  Risk  Management  Plan  as  developed  in  Sub-process  7. 
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f)  Perform  data  and  document  management  in  accordance  with  the  data  and  document  management 
plan.  In  doing  this  activity,  the  following  tasks  should  be  considered  in  accordance  with  the  data  and 
document  management  plan  (Sub-process  7). 

1)  Capture  and  organize  inputs  as  well  as  current,  intermediate,  and  final  outputs. 

2)  Provide  data  correlation  and  traceability  among  requirements,  designs,  solution,  decisions,  and 
rationale. 

3)  Be  responsive  to  established  configuration  management  procedures. 

4)  Function  as  a  reference  and  support  tool  for  the  systems  engineering  effort. 

5)  Make  data  available  and  shareable  as  called  out  in  the  contract  or  with  other  agreements. 

g)  Manage  the  information  database  to  ensure  that  captured  data  is  properly  retained,  is  secure,  and  is 
available  to  those  with  authority  to  have  access. 

Managing  the  information  database  includes  setting  up  appropriate  databases  and  procedures  for 
capturing  and  retaining  design  data  and  schema,  tools,  and  models.  Data  pertinent  to  the  technical 
effort  are  readily  accessible  and  should  be  maintained  throughout  the  system  life  cycle.  Safeguards  are 
implemented  to  ensure  data  integrity  and  security  and  to  prevent  inadvertent  loss  or  modification  of 
data.  The  program  has  the  responsibility  to  assure  that  the  data  is  collected,  stored,  controlled,  and 
available  for  proper  configuration  management  of  the  evolving  product  design,  specifications,  and 
baseline.  All  data  products  should  be  received,  logged,  archived,  recovered,  transmitted,  and 
distributed  as  required.  In  doing  this  activity,  the  following  tasks  should  be  considered: 

1)  Review  data  management  activities  periodically  to  confirm  that  the  program  data  requirements  are 
still  valid. 

2)  Ensure  that  the  process  for  review,  approval  and  release  of  data  is  well  understood  through  the 
program. 

3)  Establish  the  capability  to  retrieve  desired  program  data  quickly. 

4)  Archive  data  efficiently  based  upon  common  characteristics  (e.g.,  key  word,  topics,  contract 
number,  etc.). 

h)  Manage  and  track  stakeholder  requirements,  system  technical  requirements,  logical  solution 
representations,  physical  solution  representations,  derived  technical  requirements,  specified 
requirements,  approved  changes,  and  validation  results. 

In  systems  with  long  development  cycles,  requirements  can  change  significantly  during  the 
development  period.  As  the  system  development  progresses,  both  users  and  developers  become  more 
knowledgeable  about  both  the  requirements  and  the  system.  This  inevitably  leads  to  changes  in  the 
requirements.  If  the  proposed  changes  are  ignored,  the  delivered  system  will  fail  to  satisfy  the  users’ 
needs.  If  the  proposed  changes  are  accepted,  cost  overruns  and  delays  usually  accompany  the 
requirement  changes.  In  most  developments,  the  decision  is  made  to  “freeze”  requirements  as  early  as 
possible,  often  resulting  in  systems  that  fail  to  meet  users’  needs.  Recognizing  that  requirements 
change  in  nearly  every  system  development,  the  problem  becomes  one  of  managing  the  changes  in  an 
efficient  manner.  These  circumstances  include  changes  in  the  external  environment,  a  better 
understanding  of  users’  needs,  or  a  better  understanding  of  development  success  and  failures.  As  in 
traditional  system  development  models,  the  team  must  balance  performance,  cost,  and  schedule  factors 
when  making  decisions  about  the  acceptance  of  new  requirements,  as  well  as  removal  of  previously 
baselined  requirements  that  have  been  overcome  by  events.  The  project  team  uses  the  Outcomes 
Management  process  as  a  basis  for  making  prudent  development  decisions.  In  the  event  that  the 
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membership  of  the  team  has  changed  significantly  since  the  development  of  the  original  requirements, 
a  likely  scenario  in  systems  with  long  development  timelines,  the  team  has  at  its  disposal  both  the 
decisions  and  rationale  that  were  previously  captured.  When  the  requirements  baseline  is  modified,  the 
rationale  associated  with  each  existing  or  new  requirement  is  also  modified,  thereby  providing 
traceability  and  history.  As  the  system  matures  in  its  development  lifecycle,  it  is  expected  that  both  the 
magnitude  and  number  of  changes  will  decrease. 

There  are  software  programs  designed  specifically  to  assist  in  the  management  and  tracking  of  the 
systems  engineering  process  such  as  DOORS,  CORE™,  SLATE™,  and  Rational  Rose.  It  is  strongly 
encouraged  that  these  are  evaluated  for  appropriateness  to  the  project  and  used  whenever  feasible. 


Outputs 

•  Program  Information  (SP  13) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Next  Processes 
Control  Process 

Sub-process  13:  Information  Dissemination 
Agents 

Program  Manager  (PM) 

Systems  Engineering 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 


Metrics  and  Measures 

Information  is  accurate  and  available  in  a  timely  manner  as  defined  by  the  program. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  help  to  ensure  that  the  outcomes  of  the  applicable  processes  for  engineering  a 
system  are  properly  recorded  and  managed  according  to  the  applicable  plan,  the  agreement,  or  enterprise 
policies  and  procedures. 


Sub-process  13  -  Information  Dissemination 

The  developer  shall  ensure  that  required  and  requested  information  is  disseminated  in 
accordance  with  the  agreement,  project  plans,  enterprise  policies,  and  enterprise  procedures. 


The  purpose  of  this  sub-process  is  to  ensure  that  required  and  requested  information  is  properly  disseminated  so 
that  necessary  communications  within  the  project  and  enterprise,  and  with  the  customer  and  other  stakeholder 
community,  are  efficiently  and  effectively  completed  throughout  the  system  life  cycle.  Project  risks  are 
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increased  when  information  is  not  available  for  decision-making  in  a  timely  manner  or  if  the  information 
provided  is  of  insufficient  quality  (e.g.,  too  much,  incomplete,  not  relevant,  or  inaccurate). 


Preceding  Process 

Information  requests  could  come  from  any  of  the  other  32  sub-processes. 

Inputs 

•  Program  Information  (SP  12) 

Enterprise  Data  Repository  (information  database)  that  consists  of  recorded  outputs  from  sub¬ 
processes  1  through  12  and  14  through  33. 

•  Requests  for  information  (SP  All)  used,  in  conjunction  with  Sub-process  12  and  the  information  from  all 
other  sub-processes,  to  determine  the  kinds  of  information  to  capture  in  the  Enterprise  Data  Repository 
Repository  or  information  database,  such  as  the  following: 

Supplier  workforce  capability,  resource  availability  and  other  legal,  regulatory,  enterprise  and  project 
bounds  to  determine  capability  to  meet  acquisition  request  requirements.  (Sub-process  1) 

Acquirer  legal,  regulatory,  enterprise  and  project  bounds  affecting  establishment  of  an  agreement. 
(Sub-process  2) 

Requirement  or  operational  concept  changes  that  might  affect  supplier’s  project.  (Sub-process  3) 
External  and  internal  legal,  regulatory,  or  directive  documents  that  could  affect  the  project.  (Sub¬ 
process  4) 

Project  requirements.  (Sub-process  5) 

Key  events,  related  tasks,  and  relevant  completion  criteria  for  the  applicable  enterprise-based  life-cycle 
phase.  (Sub-project  6) 

Previously  completed  and  approved  technical  plans.  (Sub-process  7) 

Work  to  be  done,  resource  sources,  schedules,  budgets,  and  reporting  requirements.  (Sub-process  8) 
Planned  process  metrics.  (Sub-process  9) 

Planned  product  metrics.  (Sub-process  10) 

Technical  Review  Plan,  effectiveness  analyses  outcomes,  risk  analyses  outcomes,  and  trade-off 
analyses  outcomes  and  assumptions.  (Sub-process  11) 

Technical  plans,  as  applicable,  for  configuration  management,  change  management,  interface 
management,  risk  management,  and  data  and  document  management.  (Sub-process  12) 

Acquirer  and  other  stakeholder  requirements.  (Sub-process  14  and  15) 

System  Technical  Requirements.  (Sub-process  16) 

Logical  solution  representations  derived  technical  requirements,  and  system  technical  requirements. 
(Sub-process  17) 

Selected  physical  solution  representation  and  associated  derived  and  system  technical  requirements. 
(Sub-process  18) 

Design  solution  work  products  including  specified  requirements  and  acquirer  input  requirements.  (Sub¬ 
process  19) 

Enabling  product,  shipping  and  storage,  site  preparations,  installation,  acceptance  and  certification 
testing,  training  and  in-service  support  requirements,  as  appropriate  to  agreement.  (Sub-process  21) 
Effectiveness  analyses  and  risk  analyses  outcomes.  (Sub-process  22) 

Characterization  of  solutions  to  be  analyzed.  (Sub-process  23) 

Acceptable  levels  of  risk  to  the  project.  (Sub-process  24) 

Requirements  from  Sub-processes  16,  17,  18  and  19.  (Sub-process  25) 

Acquirer  requirements  sources  (inputs  to  Sub-process  14)  and  set  of  defined  acquirer  requirements 
(outputs  of  Sub-process  14). 

Other  stakeholder  sources  (inputs  to  Sub-process  15)  and  the  set  of  defined  other  stakeholder 
requirements  (output  of  Sub-process  15). 

Stakeholder  requirements  (inputs  to  Sub-process  1 6)  and  the  set  of  defined  system  technical 
requirements  (outputs  from  Sub-process  16). 

System  technical  requirements  (inputs  to  Sub-process  17)  and  sets  of  logical  solution  representations 
and  derived  technical  requirements  (outputs  of  Sub-process  17). 

Requirements  for  the  selected  physical  solution  representation  (inputs  to  Sub-process  19)  and  the 
physical  solution  specified  requirements  (outputs  from  Sub-process  19). 

Physical  solution  working  products  including  specified  requirements  (outputs  of  Sub-process  19). 
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Requirements  for  enabling  products  (output  of  Sub-process  19). 

Acquirer  requirements  (output  from  Sub-process  14). 

Trade-off  analysis/ Analysis  of  Alternatives  recommendations,  impacts  and  assumptions  (outputs  of 
Sub-process  23). 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  appropriate  agents. 

The  information  requested  from  the  Enterprise  Data  Repository  (information  database)  are  certified  as  being  up- 
to-date,  accurate,  reliable,  and  releasable  by  an  appropriate  agent. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  Information  to  consider 
for  dissemination  includes,  as  appropriate,  the  materials  captured  and  controlled  in  the  information  database. 
Tasks  to  consider  include  a)  though  j)  in  the  following,  and  tasks  to  complete  include  k)  through  s): 

a)  Provide  technical  progress  status: 

Architecture  products  as  defined  in  the  C4ISR  Architecture  Framework  and  Joint  Technical 
Architecture  (JTA). 

Process  and  product  metric  data  resulting  from  Sub-processes  9  and  10  should  be  disseminated  to  meet 
approved  requests  and  as  specified  in: 

•  Project  agreements  (Sub-processes  1  and  2)  and  task  assignments  (Sub-process  8). 

•  Project  plans,  especially  project  technical  plans  such  as  the  SEP  or  engineering  plan  (Sub-process 

7). 

•  Enterprise  policies  and  procedures. 

b)  Provide  technical  planning  information. 

Appropriate  technical  plans  and  work  packages  (Sub-processes  7  and  8)  should  be  disseminated  to 
project  teams  and  other  required  or  approved  recipients. 

c)  Disseminate  approved  and  controlled  requirements. 

Acquirer,  other  stakeholder,  system  technical  and  derived  technical  requirements,  and  all  changes  to 
requirements,  (Sub-process  14,  15,  16,  17,  18,  19  and  12)  should  be  distributed  in  a  timely  manner  to 
all  stakeholders  to  ensure  that  all  work  is  conducted  in  accordance  with  the  latest  approved 
requirements. 

Two  types  of  output  specified  requirements  are  Performance  Specifications  and  Detail  Specifications. 
These  requirements  are  used  for  realizing  the  end  product  and  are  allocated  to  subsystems  of  the  end 
product  for  developing  lower  level  building  blocks.  As  descriptions  of  the  end  product  solution,  they 
are  also  used  for  product  verification  (Sub-process  31). 

•  Performance  specifications  are  used  when  it  is  appropriate  to  state  requirements  in  terms  of:  (1) 
the  required  results  without  stating  the  method  for  achieving  the  required  results;  (2)  function 
(what  is  to  be  accomplished)  and  performance  (how  well  each  function  is  to  be  performed);  (3)  the 
environment  in  which  the  product(s)  must  perform  these  functions;  (4)  the  interface  and 
interchangeability  characteristics;  and  (5)  the  means  for  verifying  compliance. 

•  Detail  specifications  are  used  when  it  is  appropriate  to  state  design  requirements  in  terms  of:  (1) 
material  to  be  used;  (2)  how  a  requirement  is  to  be  achieved;  and  (3)  how  a  product  is  to  be 
fabricated  or  constructed. 

d)  Provide  information  for,  and  from,  technical  reviews. 
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As  appropriate,  the  following  (Sub-process  11)  should  be  disseminated  to  approved  recipients  and  as 
specified  in  the  agreement,  technical  review  plan,  and  enterprise  policies  and  procedures: 

•  Read-ahead  technical  review  package  to  technical  review  board  members. 

•  Information  and  items  necessary  to  demonstrate  that  event-based  criteria  have  been  satisfied  for 
initiation  of  the  review. 

•  Information  packages  and  presentation  materials  at  the  review. 

•  Minutes  of  the  review. 

•  Action  items  required  for  closure. 

•  Final  review  closeout  approval. 

•  Technical  Review  Report 

e)  Make  available  design  data  and  schema. 

Data  pertinent  for  the  technical  effort  (Sub-processes  17,  18  and  19)  should  be  disseminated  to  project 
teams  and  team  members  to  ensure  information  availability  for  decisions  and  events  and  to  other 
authorized  recipients  requesting  information. 

Design  data  and  schema  information  should  include,  as  appropriate,  source,  version,  and  distribution 
information  for  documents  used  in  the  engineering  or  reengineering  of  system  products  and  services 
including  system  product  technical  data  packages.  The  technical  data  package  should  consist  of,  as 
appropriate:  a  buy-to  description  (e.g.,  detail  specifications  and/or  final  drawings);  a  build-to 
description  (models,  final  drawings,  and  detail  or  performance  specifications  depending  on  the 
maintenance  concept,  production  plan,  tool  design,  bill  of  materials,  and  statistical  process  control 
plan);  design  documentation;  engineering  changes,  deviations,  and  waivers;  and  enabling  product 
descriptions. 

f)  Make  available  lessons  learned. 

Lessons  learned  from  applicable  sub-process  implementation  that  have  been  recorded  in  the  Enterprise 
Data  Repository,  or  other  lessons  learned  document,  should  be  disseminated  to  other  projects  within 
the  enterprise,  to  other  teams  within  the  project,  and  to  project  suppliers  as  appropriate. 

g)  Report  variances. 

Product  and  process  variances  and  anomalies  (Sub-processes  9  and  10,  and  25  through  33  (progress 
assessments,  validations  and  verifications))  should  be  reported  along  with: 

•  Recommended  actions  to  return  the  product  or  process  metric  to  established  expectations  or 
requirements. 

•  Cost  and  schedule  impacts. 

•  Effect  on  the  project  if  action  is  not  taken. 

h)  Disseminate  data  deliverables. 

Data  deliverables  generated  by  project  sub-processes  should  be  disseminated  as  required  by  the 
agreement,  enterprise  policies  and  procedures,  and  project  plans  including  the  engineering  plan. 

i)  Disseminate  approved  changes. 

Approved  requirements  and  design  changes  (Sub-process  12)  and  updated  plans  (Sub-processes  5,  6 
and  7)  should  be  distributed  to  approved  or  required  recipients. 

j)  Disseminate  directives. 

Work  directives  resulting  from  management  decisions  (Sub-processes  1 1  and  12),  planning  (Sub¬ 
processes  4  through  8),  and  approved  changes  (Sub-process  12)  should  be  disseminated  to  intended 
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recipients  that  will  initiate  or  change  work  by  project  teams  or  support  organizations  within  the 
enterprise. 


In  addition  to  the  tasks,  the  following  tasks  should  be  completed: 

k)  Establish  a  framework  for  information  flow  within  the  project  including  the  language(s)  to  be 
employed  in  project  information  exchanges. 

l)  Maintain  an  information  library  or  reference  index  to  provide  information  available  and  access 
instructions. 

Access  information  should  include  means  of  access,  access  security  passwords,  time  period 
information  will  be  available,  and  personnel  cleared  for  access.  This  is  to  allow  direct  access  to  the 
Enterprise  Data  Repository  for  those  persons  with  access  authority  and  who  have  the  technology 
available  to  enable  access. 

m)  Identify  and  document  the  data  delivery  requirements  found  in  the  agreement,  project  plans  and 
enterprise  policies  and  procedures. 

Requirements  include  information  desired,  when  required,  scope  of  information  to  be  made  available, 
security  and  special  handling,  metrics,  summaries,  change  control,  traceability,  and  delivery 
instructions. 

n)  Establish  a  handling,  approval  and  disposition  procedure  for  identified  data  deliverables. 

o)  Establish,  as  appropriate,  a  data/information  request  form  and  a  handling,  approval,  and  disposition 
procedure  for  special  requests  for  project  information. 

p)  Assign  appropriate  responsibilities  and  authorities  to  persons  or  groups  for  the  handling,  approval  and 
disposition  of  received  information  requests  and  identified  data  deliverables  from  the  agreement, 
project  plans,  and  enterprise  policies  and  procedures. 

Persons  and  groups  assigned  responsibility  and  authority  to  disseminate  data  and  information  should 
be  informed  of  their  obligations  and  responsibilities,  especially  with  respect  to  information  and  data 
legislation,  security,  privacy,  ownership,  agreement  restrictions,  rights  of  access,  intellectual  property, 
copyrights,  and  patents. 

q)  Set  up  a  data  delivery  system  to  control  what  has  to  be  delivered,  when  it  has  to  be  delivered,  the 
format  of  the  data  to  be  delivered,  the  medium  in  which  the  data  is  to  be  delivered,  delivery  status,  and 
any  other  peculiar  handling,  storage  or  classification  of  the  data  required. 

Information  may  originate  and  may  terminate  in  any  form  (e.g.,  verbal,  textual,  graphical,  numerical) 
and  may  be  stored,  processed,  replicated,  and  transmitted  using  any  medium  (e.g.,  electrical,  printed, 
magnetic,  or  optical). 

Relevant  information  storage,  transformation,  transmission  and  presentation  standards  and  conventions 
should  be  used  according  to  agreements,  legislation  constraints,  and  enterprise  policy. 

The  status  of  information  items  disseminated  (e.g.,  version  description,  record  of  distribution,  security 
classification,  recipient,  authority  for  dissemination,  end  product  approving  agent)  should  be  recorded. 

r)  Evaluate  the  information  system  to  identify  generation  and  recording  of  performance  issues  and 
problems;  application  of  information  in  the  current  system  life-cycle  stage;  satisfaction  of  information 
users;  risks  associated  with  delayed  or  corrupted  information,  unauthorized  access,  or  survivability  of 
information  from  hazards  such  as  fire,  flood,  earthquake,  etc.;  and  recommend  improvements. 
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Evaluation  should  include:  (1)  proof  of  correctness,  accessibility,  availability,  reliability,  and  security 
of  data/information  provided  to  internal  and  external  recipients;  and  (2)  proof  of  coherence  of  the 
overall  project  information  set  to  facilitate  effective  and  efficient  use  of  the  information  both  during 
and  after  the  project. 

s)  Assure  that  required  and  requested  information  is  appropriately  distributed  to  satisfy  the  needs  of  the 
acquirer  and  requesters  in  accordance  with  the  agreement,  project  directives  and  plans,  and  enterprise 
policies  and  procedures. 

Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Completed  request  for  information  forms  (SP  12) 

•  Status  of  Information  dissemination  (SP  12) 

•  Program  information  (SP  All)  to  be  delivered  to  the  requesting  sub-process  as  required  by  the  agreement, 
project  plans  including  the  engineering  plan  (SEP),  and  enterprise  policies  and  procedures,  as  well  as 
required  by  appropriately  approved  requests.  Example  outputs  include: 

Agreements 

Directives  to  do  work  (e.g.,  task  assignments,  and  work  authorizations) 

Information  for  doing  work  (e.g.,  agreement  tasks;  requirements;  schedules;  budget  allocations; 
product  interfaces  -  physical,  data,  human,  functional;  and  work  interfaces  -  other  teams,  other 
projects,  other  organizations) 

Explanations  for  work  done  (e.g.,  rationale  for  design  decisions) 

Recommendations  including  assumptions  made  with  respect  to  trade-off  analyses 
Sources  of  information  (e.g.,  websites,  standards,  or  directives) 

Best  practices  used  in  the  technical  work  of  the  project  (e.g.,  tools,  and  methods) 

Status  information  (e.g.,  progress,  issues,  risks,  variations  and  actions  being  taken  with  expected 
results) 

Cost,  schedule  and  performance  constraints  and  thresholds 

WBS  information 

IMS  (Integrated  Master  Schedule) 

IMP  (Integrated  Master  Plan) 

Enabling  product  information  (e.g.,  requirements  for  development  or  for  acquisition  of  existing 
enabling  products) 

Approved  changes 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

•  Recipients  are  authorized  to  have  the  information  and  have  the  proper  security  clearances  to  receive  the 
information  when  it  is  classified. 

•  Information  is  properly  packaged,  handled,  shipped/transmitted,  and  controlled  as  appropriate  to  the 
classification  and  sensitivity  of  the  material  being  disseminated. 

Next  Processes 

Sub-process(es)  corresponding  to  the  requested  information. 

Agents 

Information  Specialist 

Data  and  Document  Manager 

Systems  Engineering 

Acquirer 

Supplier 

Tools 

Microsoft  Word 
Excel  Spreadsheet 
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Enterprise  Data  Repository 


References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Organizational  Environment  For  Integration 
process  areas 

Security  control  directives  for  the  handling,  packaging  and  transmittal  of  classified  information 
Metrics  and  Measures 

Percent  of  on-time  deliveries  of  information  requested. 

Percent  of  on-time  deliveries  of  information  required. 

Number  of  complaints  on  the  quality  of  disseminated  information. 

Number  of  security  violations  for  improper  handling,  storage,  and  transmittal  of  classified  materials. 

The  expected  outcomes  for  the  representative  tasks  associated  with  this  distribution  are  provided  in  Appendix 
C.  The  outcomes  associated  with  completing  this  sub-process  help  to  ensure  that  the  required  and  requested 
information  is  appropriately  distributed  to  satisfy  the  needs  of  the  acquirer  and  requesters,  in  accordance  with  an 
agreement,  project  directives  and  plans,  and  enterprise  policies  and  procedures. 


4.3  System  Design 

The  System  Design  Processes  are  used  to  convert  agreed-upon  requirements  of  the  acquirer  into  a  set  of 
realizable  products  that  satisfy  acquirer  and  other  stakeholder  requirements. 

Two  processes  are  involved  -  Requirements  Definition  and  Solution  Definition.  The  relationship  of  these 
processes  is  shown  in  Figure  4.3a. 
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Acquirer  and  Other  Stakeholder  Requirements 


The  systems  design  process  is  a  top-down  comprehensive,  iterative  and  recursive  problem  solving  process 
applied  sequentially  through  all  Life  Cycle  Phases  and  Stages  of  Development  as  shown  in  Figure  4.3b,  which 
is  from  DoD5000  Instructional  Information: 


User  Needs  &  Technology  Opportunities 
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Figure  4.3b  -  System  Design  Process 
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During  the  Stages  of  Development,  the  iterative  process  is  used  to: 

•  transform  needs  and  derived  requirements  into  a  set  of  system  product  and  process  descriptions 
(adding  value  and  more  detail  with  each  level  of  development); 

•  generate  information  for  decision  makers;  and 

•  provide  input  for  the  next  level  of  development. 

As  illustrated  by  the  System  Design  Relational  Diagram,  the  fundamental  systems  design  activities  are: 
Acquirer  and  Stakeholder  Requirements  Definition,  System  Technical  Requirements  Definition,  Logical 
Solutions  Representation  (Functional  Analysis  and  Allocation),  Physical  Solution  Representation  (Design 
Synthesis),  and  Specified  Requirements  Definition;  all  balanced  by  other  processes  within  this  Guide  called 
Assessment,  Control,  and  System  Analysis.  These  processes  are  used  to  make  decisions  and  track 
requirements,  maintain  technical  baselines,  manage  interfaces,  identify  and  manage  risks,  track  cost  and 
schedule,  track  technical  performance,  verify  requirements  are  met,  and  review/audit  the  progress. 

During  system  design  iteration,  derived  requirements  and  architectures  are  generated  to  better  describe  and 
understand  the  system.  The  word  “architecture”  is  used  in  various  contexts  in  the  general  field  of  engineering. 

It  is  used  as  a  general  description  of  how  the  sub-systems  join  together  to  form  the  system.  It  can  also  be  a 
detailed  description  of  an  aspect  of  a  system:  for  example,  the  operational,  system,  and  technical  architectures 
used  in  hardware  and  software  intensive  developments.  However,  systems  engineering  management,  as 
developed  in  DoD,  recognizes  three  universally  usable  architectures  that  describe  important  aspects  of  the 
system:  functional,  physical,  and  system  architectures. 

The  functional  architecture  identifies  and  structures  the  allocated  functional  and  performance  requirements. 

The  physical  architecture  depicts  the  system  product  by  showing  how  it  is  broken  down  into  subsystems  and 
components.  The  system  architecture  identifies  all  the  products  (including  enabling  products)  that  are 
necessary  to  support  the  system  and,  by  implication,  the  processes  necessary  for:  development, 
production/construction,  deployment,  operations,  support,  disposal,  training,  and  verification. 

Life  Cycle  Phase  integration  is  achieved  through  integrated  development  -  that  is,  concurrent  consideration  of 
all  life  cycle  needs  during  the  development  process.  DoD  policy  requires  integrated  development  to  be 
practiced  at  all  levels  in  the  acquisition  chain  of  command  as  described  in  the  Integrated  Product  and  Process 
Development  (IPPD)  Handbook.  Concurrent  consideration  of  all  life  cycle  needs  can  be  greatly  enhanced 
through  the  use  of  interdisciplinary  teams.  These  teams  are  often  referred  to  as  Integrated  Product  Teams  (IPT). 
The  objective  of  an  IPT  is  to: 

•  produce  a  design  solution  that  satisfies  initially  defined  requirements,  and  communicates  that  design 
solution  clearly,  effectively,  and  in  a  timely  manner; 

•  place  balanced  emphasis  on  product  and  process  development; 

•  assure  early  involvement  of  all  disciplines  appropriate  to  the  team  task;  and 

•  achieve  concurrent  technical  management. 

Life-cycle-phase  functions  are  the  characteristic  actions  associated  with  the  system  life  cycle.  They  are 
development,  production  and  construction,  deployment  (fielding),  operation,  support,  disposal,  training,  and 
verification.  These  activities  cover  the  “cradle  to  grave”  life  cycle  process.  The  customers  of  systems  design 
perform  the  life  cycle  functions.  The  system  user’s  needs  are  emphasized  because  their  needs  generate  the 
requirement  for  the  system,  but  it  must  be  remembered  that  all  of  the  life-cycle-phase  functional  areas  generate 
requirements  for  the  system  design  once  the  user  has  established  the  basic  need.  Those  that  perform  these 
functions  also  provide  life  cycle  representation  in  design-level  integrated  teams. 

This  technical  effort  begins  with  identifying,  collecting,  and  defining  acquirer  and  other  stakeholder 
requirements.  These  requirements  are  transformed  into  a  set  of  validated  system  technical  requirements.  The 
validated  system  technical  requirements  are  then  transformed  into  a  design  solution  described  by  a  set  of 
specified  requirements.  The  specified  requirements  take  the  form  of  specifications,  drawings,  models,  or  other 
design  documents  depending  on  design  maturity.  These  are  used  to:  (1)  build,  code,  assemble  and  integrate  end 
products;  (2)  verify  end  products  against  requirements;  (3)  obtain  off-the-shelf  products;  or  (4)  assign  to  a 
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supplier  the  development  of  subsystem  products.  The  relationship  between  the  requirements  involved  with  the 
System  Design  Processes  is  shown  in  Figure  4.3c. 


NOTE  -  Requirements  traceability  is  instituted  for  tracking  requirements  from  the  identification  of  acquirer  and  other 
stakeholder  requirements  to  the  system  technical  requirements  logical  solution  representations,  physical  solution 
representations,  derived  technical  requirements,  and  specified  requirements.  (See  Sub-process  12.  task  h).) 
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System  Design  Relational  Diagram 


Source:  ANSI/EIA-632-1998 


Figure  4.3c  -  System  design  relation  diagram 


67 


Sub-process  14 


4.3.1  Requirements  Definition  Process 

The  three  sub-processes  associated  with  the  Requirements  Definition  Process  are  shown  in  Figure  4.3.1a. 


Sub-process  14  -  Acquirer  Requirements 


Requirements 

Definition 

Process 


Sub-process  15  -  Other  Stakeholder  Requirements 


System  DesigfK  Requirements 


Sub-process  16  -  System  Technical  Requirements 


Figure  4.3.1a  -  Requirements  Definition  Process/Sub-processes 


Inputs  to  the  Requirements  Definition  Process  are  of  three  types:  (1)  requirements  from  the  agreement,  other 
documents,  and  individuals  or  groups  that  have  a  stake  in  the  outcome  of  the  engineering  or  reengineering  of  the 
system,  (2)  requirements  in  the  form  of  outcomes  from  other  processes  such  as  technical  plans  and  decisions 
from  technical  reviews,  and  (3)  requested  or  approved  changes  to  requirements  of  the  first  type. 

The  Department  of  Defense  (DoD)  inputs  to  this  process  are  the  Initial  Capabilities  Document  (ICD)  -  formerly 
Mission  Needs  Statement  (MNS),  and  Capability  Development  Document  (CDD)  -  formerly  Operational 
Requirements  Document  (ORD).  These  items  are  well  defined  for  formal  Acquisition  Category  (AC AT) 
programs  but  should  be  completed  on  an  informal  level  for  all  programs.  These  should  be  reviewed  for 
appropriateness  repeatedly  throughout  this  process  as  the  product  evolves. 


NOTES 


1  The  requirements  defined  by  this  process  come  from  stakeholders  who  have  an  interest  in  the  system 
being  engineered.  Stakeholders  are  of  two  kinds:  the  acquirer  of  the  system  products  (see  the  definition  of 
acquirer  in  the  Glossary,  Appendix  A)  and  all  other  stakeholders  (see  the  definition  of  other  stakeholders 
in  Appendix  A). 

2  The  Requirements  Definition  Process  is  used  to  transform  stakeholder  requirements  into  a  set  of 
system  technical  requirements.  These  requirements  are  stated  in  acceptable  technical  terms  and  represent 
a  reasonably  complete  description  of  the  problem  that  must  be  solved  to  provide  a  set  of  end  products  and 
enabling  products  that  meet  the  acquirer’s  and  other  stakeholders’  needs  and  expectations. 

3  The  Requirements  Definition  Process  is  re-accomplished,  as  necessary,  whenever  requirements  in  an 
agreement  change  or  when  other  stakeholder  requirements  are  identified  that  affect  the  product  design  or 
otherwise  constrain  the  technical  effort  required  to  engineer  a  new  system,  develop  a  derivative  system,  or 
reengineer  a  legacy  system.  Such  changes  could  be  caused  by  technology  limitations,  project  schedule 
and  cost  anomalies,  or  new  requirements. 

4  Sometimes  it  is  important  to  preserve  competition  when  defining  requirements  to  ensure  that  there  will  be  more 
than  one  supplier  that  can  meet  the  requirements.  Otherwise,  the  cost  of  a  single  supplier  can  be  too  high  since  there 
can  sometimes  be  little  incentive  to  give  a  low-cost  bid. 


Sub-process  14  -  Acquirer  Requirements 

The  developer  shall  define  a  validated  set  of  acquirer  requirements  for  the  system,  or  portion 
thereof. 


Preceding  Processes 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Requirements  Validation  Process 

Sub-process  26:  Acquirer  Requirements  Validation 
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Inputs  (“EXT”  indicates  it  is  external,  unspecified,  and  not  from  a  sub-process.) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Need  Statement  (MNS)  (User,  Fleet)  (EXT) 

•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD) 
(OPNAV)  (EXT) 

•  Engineering  Investigation  Reports  (In-Service,  Safety,  Logistics,  etc.)  (User,  Fleet)  (EXT) 

•  Utilization  and  Readiness  Reports  (NALCOMIS)  (EXT) 

•  Specifications  from  higher  level  system  building  blocks  (EXT) 

•  Sponsor  High-Level  Operational  Concept  Graphic  (OV-1)  architecture  (EXT) 

•  Effectiveness  Analysis  Reports  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Acquirer  Requirements  Validation  Revisions  (SP  26) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Tasks 

The  team  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Identify,  collect,  and  prioritize  assigned,  customer,  user,  or  operator  requirements  for  the  system, 
or  portion  thereof,  including  any  requirements  for  development,  production,  test, 
deployment/installation,  training,  operations,  support/maintenance,  and  disposal  of  the  system’s 
products. 

The  expected  input  from  the  sponsor  should  include: 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Need  Statement  (MNS) 

•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD) 

•  Program  objectives 

•  Mission  Area  Analysis  (MAA)  (Sub-process  22:  Effectiveness  Analysis) 

•  Measures  of  Effectiveness  (MOE)  (Sub-process  22:  Effectiveness  Analysis) 

•  High-Level  Operational  Concept  Graphic  (OV-1)  architecture 

Although  the  sponsor  typically  provides  these  inputs,  analyses  and  validation  are  required  to  ensure  the 
team  has  a  clear  understanding  of  the  customer  requirements.  In  cases  were  these  documents  are  not 
provided,  the  team  shall  perform  appropriate  modeling,  simulation,  and  analysis  to  develop  comparable 
requirements  studies.  These  analyses  include: 

•  Surveying  the  sponsor,  fleet  operators,  and  maintainers 

•  Mission  analysis  (Sub-process  22:  Effectiveness  Analysis) 

•  System  concept  analysis  (Sub-process  22:  Effectiveness  Analysis) 

•  Operational  concept  analysis  (Sub-process  22:  Effectiveness  Analysis) 

•  Operational  requirements  analysis 

b)  Ensure  that  the  resulting  set  of  requirements  agrees  with  the  acquirer  needs  and  expectations  (see 
Sub-process  26). 

c)  Record  the  resulting  set  of  acquirer  requirements  in  the  established  information  database  (see 
Sub-process  12). 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  2,  4,  7,  10,  11,  16,  31, 
33) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Measures  of  Effectiveness  (MOE)  (SP  5,  7,  16) 
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•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD)  (SP  2, 
4,7,10,11,16,31,33) 

•  Specifications  from  higher  level  system  building  blocks  (SP  1 6) 

•  Acquirer  requirements  (SP  5,  16,  26) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  4:  Process  Implementation  Strategy 
Sub-process  5:  Technical  Effort  Definition 
Sub-process  7:  Technical  Plans 
Assessment  Process 

Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Requirements  Validation  Process 

Sub-process  26:  Acquirer  Requirements  Validation 
System  Verification  Process 

Sub-process  31:  End  Product  Verification 
End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Agents 

Acquirer 

User 

Concepts  Analysis 

Cost  Analysis 

Fleet  Project  Team  (FPT) 

Operations  (Ops)  Analysis 
R&M 

Systems  Engineering 

Tools 

Survey 

Questionnaire 

Quality  Function  Deployment  (QFD)  Capture 
Modeling  &  Simulation  (M&S) 

Queuing  Methodology  (AWESim,  SLAM) 

Integrated  Definition  (IDEF) 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 
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•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
Systems  Engineering  &  Analysis  (Blanchard) 

MIL-STD-498 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Requirements  Development  process  areas 


Metrics  and  Measures 

Percent  completion  of  analysis  and  output  products. 

Percent  of  acquirer  requirements  that  have  been  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  are  used,  when  combined  with  other  stakeholder  requirements,  to  define  the 
system  technical  requirements,  and  to  identify  requirements  for  enabling  products. 


Sub-process  15  -  Other  Stakeholder  Requirements 

The  developer  shall  define  a  validated  set  of  other  stakeholder  requirements  for  the  system, 
or  portion  thereof. 


Preceding  Process 
Planning  Process 

Sub-process  4:  Process  Implementation  Strategy 
Sub-process  8:  Work  Directives 
Systems  Analysis  Process 

Sub-Process  22:  Effectiveness  Analysis 
Requirements  Validation  Process 

Sub-process  27:  Other  Stakeholder  Requirements  Validation 

Inputs  (“EXT”  indicates  it  is  external,  unspecified,  and  not  from  a  sub-process.) 

•  List  of  stakeholders  and  roles  (SP  4) 

•  Team  Work  Plan  (TWP)  (SP  8) 

•  Statement  of  Objectives  (SOO)  (SP  8) 

•  Statement  of  Work  (SOW)  (SP  8) 

•  Effectiveness  Analysis  Reports  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Other  stakeholder  requirements  validation  revisions  (SP  27) 

•  DoD/Naval  policy  and  directives  (EXT) 

•  Federal/Intemational  Laws  and  regulation  (EXT) 

•  International  /National  standards  (EXT) 

•  Team  /  Project  objectives,  constraints,  and  policy  (EXT) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Identify  and  collect  other  stakeholder  requirements  that  can  constrain  the  system’s  end 
products.  Be  sure  to  consider  joint  project  stakeholders  requirements. 
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b)  Identify  and  collect  other  stakeholder  requirements  that  can  constrain  development,  production, 
test,  deployment/installation,  training,  support/maintenance,  and  disposal  of  the  system 
products. 

c)  Identify  and  collect  other  stakeholder  constraints  such  as  applicable  laws  and  regulations; 
technology  base;  standards  and  specifications;  competitor’s  product  capabilities  and  trends;  and 
interfaces  with  other  evolving  systems  or  platforms. 

d)  Ensure  that  the  resulting  set  of  requirements  agrees  with  other  stakeholder  needs  and 
expectations  (see  Sub-process  27). 

e)  Record  the  resulting  set  of  stakeholder  requirements  in  the  established  information  database  (see 
Sub-process  12). 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Other  stakeholder  requirements  (SP  5,  16,  27),  such  as: 

-  Project  plans,  Teams  (possible  Joint  Team  Projects),  Organization,  Automated  tools  metrics, 
Management  decision  criteria,  Standards,  Guides,  Policies,  Procedures,  and  Physical/fmancial 
resources 

-  Manufacturing,  Production,  Test,  Deployment,  Installation,  Training,  Support,  Disposal  processes  and 
capacities 

-  National  and  international  standards,  Laws,  Regulations,  Environment,  Technology  base,  Industry 
standards,  General  specifications,  and  Competitor  capabilities 

-  Interfaces  with  other  systems  and  platforms 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Systems  Analysis  Process 

Sub-Process  22:  Effectiveness  Analysis 
Requirements  Validation  Process 

Sub-process  27:  Other  Stakeholder  Requirements  Validation 

Agents 

Systems  Engineering 

Enterprise  Management 

Manufacturing 

PM 

PEO 

Test  &  Evaluation 

Logistics 

Depot 

Other  Systems  Commands  (Syscoms) 

Tools 

Surveys 
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Questionnaire 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
IPPD  Handbook 


Metrics  and  Measures 

Percent  completion  of  analysis  and  output  products. 

Percent  of  other  stakeholder  requirements  that  have  been  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  the  completion  of  this  sub-process  help  to  ensure  that  the  other  stakeholder  requirements  reflect  the 
interests  of  those  who  have  a  stake  in  the  outcome  of  the  project  and,  when  combined  with  acquirer 
requirements,  can  be  used  to  define  system  technical  requirements  and  requirements  for  enabling  products. 


NOTES 

1  In  general,  other  stakeholder  requirements  place  constraints  on  the  system  development,  both  on  the  resulting 
system  and  the  processes  for  developing  the  system  products. 

2  Some  sources  of  other  stakeholder  requirements  include  the  agreement,  owners  of  associated  processes, 
external  system  interfaces,  market  research,  government  and  industry  regulations,  international  conventions  and 
agreements,  projects  and  enterprise  directives,  project  and  enterprise  process  constraints,  lessons  learned,  and 
interviews. 

3  It  is  usually  not  possible  to  meet  all  other  stakeholder  requirements  for  a  particular  system  since  various 
stakeholders  (including  the  acquirer)  have  conflicting  requirements  relative  to  one  another.  Some  of  these 
requirements  can  be  addressed  in  later  versions  of  the  system. 

4  Constraints  can  result,  for  example,  from  treaties,  laws,  regulations,  standards,  culture,  natural  laws,  or  firm 
customer  or  user  needs. 

5  Constraints  also  apply  to  those  characteristics  necessary  to  interface  with  other  existing  systems. 
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Sub-process  16  -  System  Technical  Requirements 

The  developer  shall  define  a  validated  set  of  system  technical  requirements. 


Preceding  Process 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Sub-process  15:  Other  Stakeholder  Requirements 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 
Sub-process  28:  System  Technical  Requirements  Validation 

Inputs  (“. EXT ”  indicates  it  is  external,  unspecified,  and  not  from  a  sub-process.) 

•  Sponsor  High-Level  Operational  Concept  Graphic  (OV-1)  architecture  (EXT) 

•  Specifications  from  higher  level  system  building  blocks  (SP  14) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD)  (SP 
14) 

•  Measures  of  Effectiveness  (MOE)  (SP  14) 

•  Acquirer  requirements  (SP  14) 

•  Other  stakeholder  requirements  (SP  15) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Trade-off  Analysis  Technical  Report  (SP  23) 

•  Requirement  statements  validation  revisions  (SP  25) 

•  System  technical  requirements  validation  revisions  (SP  28) 

•  Technical  Data  Package  (TDP)  (SP  5) 

•  Technology  Roadmap  (SP  5) 

•  Life  Cycle  Support  Plans  (SP  5) 

•  Pre-Plan  Product  Improvement  (P3I)  (SP  5) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

Requirements  analysis  is  a  verification  of  the  system  requirements.  This  verification  can  be  provided  in  a 
System  Requirements  Document  (SRD)  from  a  system-designer  perspective.  The  intent  is  to  verify  the 
requirements  provided,  identify  over-stated  or  unnecessary  requirements,  and  to  identify  missing  requirements. 
Analysis  of  the  intended  system  operation  as  represented  in  the  Operational  Concept  Document  along  with 
analysis  of  requirements  provided  in  the  Capability  Development  Document  are  the  keys  to  identification  of 
system-level  requirements.  The  process  leads  to  the  generation  of  system-level  technical  requirements. 

Prior  analyses  shall  be  reviewed  and  updated,  refining  mission  and  environment  definitions  to  support  system 
definition.  Requirement  analysis  shall  be  conducted  to  derive  functional,  performance  and  other  requirements 
that  will  guide  system  definition  and  implementation,  and  verify  that  customer  needs  will  be  satisfied.  In 
conducting  requirement  analysis,  the  following  tasks  shall  be  performed: 
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•  Assist  in  refining  customer  objectives  and  requirements.  Provide  a  detailed  description  of  operation, 
defining  all  external  interfaces  and  system  reaction  to  input  over  these  interfaces  (including  mode 
transitions),  driving  timelines,  and  operating  environments.  Derive  first-level  functional  and  specialty 
requirements. 

•  Define  initial  performance  objectives  and  refine  them  into  requirements.  Define  performance  aspects  of  all 
functional  requirements  as  derived  from  system  operation  and  mission  timelines.  Define  MOPs,  associate 
them  with  MOEs,  and  cite  critical  Technical  Performance  Measurements  (TPMs). 

•  Flesh  out  the  system  description  by  defining  operator  involvement,  design  and  technology  constraints, 
function  concurrency  and  translation  into  capacity  requirements.  Identify  and  define  constraints  that  limit 
solutions  (e.g.,  missions  and  utilization  environments  or  adverse  impacts  on  natural  and  human 
environments). 

•  Identify  high-risk  elements  (potential  show  stoppers)  in  areas  of  cost,  performance,  and  schedule. 

Challenge  questionable  and  conflicting  requirements. 

Establishing  a  total  set  of  system  requirements  is  a  complex,  time-consuming  task  involving  nearly  all  program 
areas  in  an  interactive  effort.  It  must  be  done  early  since  it  forms  the  basis  for  all  design,  manufacturing,  test, 
operations,  maintenance,  and  disposal  efforts,  and  therefore  determines  the  cost  and  schedule  of  the  program. 

The  input  and  output  summary  tables  define  the  expected  input  and  output  for  each  of  the  above  tasks.  Output 
consists  of  both  requirements  and  design  information.  A  database  serves  as  the  point  of  capture  of  both 
categories  of  information.  System  requirements  can  also  be  documented  in  the  System/Subsystem  Specification 
(DI-IPSC-81431T  if  the  project  warrants  requirement  documentation  at  this  time. 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Establish  required  transformation  rules,  priorities,  inputs,  outputs,  states,  modes,  and 
configurations  that  will  influence  and  affect  the  other  tasks  for  definition  of  system  technical 
requirements  by  identifying  and  defining  them,  as  appropriate  to  each  system  product. 

Review  concept  of  operations  and  elaborate  where  necessary  on  describing  system  behavior,  starting 
with  outputs  generated  by  external  systems  (modified  as  appropriate  by  passing  through  the  natural 
system  environment)  which  act  as  stimuli  to  the  system,  causing  it  to  take  specified  actions  and 
produce  outputs  which  are  absorbed  by  external  systems.  These  single  threads  of  behavior  are  traced 
from  source  document  statements  and  cover  every  aspect  of  operational  performance,  including 
logistical  modes  of  operation,  operation  under  designated  conditions,  and  behavior  required  when 
experiencing  mutual  interference  with  multi-object  systems. 

Aggregation  of  these  single  threads  of  behavior  is  a  more  or  less  mechanical  process  depending  on  the 
level  of  sophistication  of  tool  support  supplied  with  the  design  decision  database.  When  aggregated, 
the  logical  sum  of  these  single  threads  of  behavior  represent  a  dynamic  statement  of  what  the  system  is 
required  to  do.  In  some  cases,  the  word  ’’scenario”  is  used  to  describe  a  single  thread  of  behavior  and 
in  other  cases  it  describes  a  superset  of  many  single  threads  operating  concurrently. 

In  defining  the  requisite  system  behavior  within  the  operating  environment(s),  transformation  rules  are 
important  in  characterizing  a  system.  A  transformation  rule  is  anything  that  tells  a  product  how  to 
transform  one  or  more  inputs  into  one  or  more  outputs  (transform  inputs  to  outputs),  or  change  from 
one  mode/state/configuration  to  another  given  certain  conditions  to  be  true  (transform  from  state  X  to 
state  Y,  for  example).  For  example: 

given  inputs  A  and  B,  produce  output  C  (inputs/outputs) 
do  the  above  only  when  in  XYZ  mode  (mode/state) 
do  the  above  only  when  in  configuration  LMN  (configuration) 
convert  A  to  A-prime  by  using  the  JKL  algorithm  (transformation  rule) 
when  both  A  and  B  received  at  same  time,  process  A  first  (priority) 
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Basically  the  nature  of  these  transformation  rules  will  differ  depending  on  the  technology  being  used, 
type  of  product  (hardware,  software,  facilities,  etc.),  or  the  standard  methods  and  tools  used  in  a 
particular  industry  or  company. 

Define  the  various  modes  of  operation  (embedded  training  capability,  full  operational,  etc.)  for  the 
system  products  under  development.  The  conditions  (environmental,  configuration,  operational,  etc.), 
which  determine  the  modes  of  operation,  are  also  defined  (IEEE  1999,  6.1.12). 

Identify  all  possible  types  of  observable  input  and  output  events  that  can  occur  between  the  system  and 
its  interacting  external  systems.  Record  them  as  input  and  output  events  in  the  database  including 
information  to  trace  the  reason  for  their  existence  to  prevent  dilution  of  originating  requirements. 

b)  Define  operational  requirements  to  include  operational  profiles,  and  for  each  operational  profile,  the 
utilization  environment,  events  to  which  system  end  products  must  respond,  frequency  of  use,  physical 
and  functional  interfaces,  and  system  functional  requirements  (what  system  end  products  must 
accomplish). 

At  the  beginning  of  the  program,  systems  engineering  is  concerned  primarily  with  operational 
requirements  analysis  -  leading  to  the  translation  of  user  needs  into  a  quantifiable  set  of  performance 
requirements  that  can  be  translated  into  design  requirements.  These  objectives  are  then  quantified  in 
broad  terms,  and  basic  functions  are  identified  that  could  fulfill  the  need.  The  objective  of  operational 
requirement  analysis  is  to  identify  and  express  technical  requirements  in  measurable  parameters  that 
state  user  needs  in  appropriate  terms  to  guide  system  concept  development.  Performing  the  mission 
analysis  in  a  parametric  manner  ensures  that  an  appropriate  system  sizing  (of  communication  links, 
data  processing  throughput  and  capacity,  number  of  computers  and  personnel,  and  facility  space)  can 
be  performed.  The  context  diagram  serves  as  a  useful  tool  to  depict  Input/Process/Output 
Requirements  analysis.  The  total  system  engineering  process  is  an  iterative  operation,  constantly 
refining  and  identifying  new  requirements  as  the  concept  develops  and  additional  details  are  defined. 

Items  1)  through  4)  below  define  information  that  should  be  included  for  each  operational  profile: 

1)  The  utilization  environment  and  factors,  natural  or  induced,  that  can  affect  end  product 
performance. 

This  task  is  to  define  the  utilization  environments  for  each  of  the  operational  scenarios.  All 
environmental  factors,  natural  or  induced,  which  may  affect  system  performance,  should  be 
identified  and  defined.  Factors  which  ensure  that  the  system  minimizes  the  potential  for  human  or 
machine  errors  or  failures  that  cause  injurious  accidents  or  death,  and  impart  minimal  risk  of 
death,  injury,  or  acute  chronic  illness,  disability,  and/or  reduced  job  performance  of  the  humans 
who  support  the  system  life  cycle,  are  identified.  Specifically,  weather  conditions  (e.g.,  rain, 
snow,  sun,  wind,  ice,  dust,  and  fog),  temperature  ranges,  topologies  (e.g.,  ocean,  mountains, 
deserts,  plains,  and  vegetation),  biological  (e.g.,  animal,  insects,  birds,  and  fungi),  time  (e.g.,  day, 
night,  and  dusk),  induced  (e.g.,  vibration,  electromagnetic,  acoustic,  and  chemical),  or  other 
environmental  factors  are  defined  for  possible  locations  and  conditions  where  the  system  may  be 
operated.  Effects  on  hardware,  software,  and  humans  should  be  assessed  for  impact  on  system 
performance  and  life  cycle  processes  (IEEE  1999,  6.1.8). 

If  the  inputs/outputs  are  expected  to  be  significantly  affected  by  the  environment  between  the 
system  and  the  external  systems,  add  concurrent  functions  to  the  context  diagram  to  represent 
these  transformations,  and  add  input  and  output  events  to  the  database  to  account  for  the 
differences  in  event  timing  between  when  it  is  emitted  to  when  it  is  received. 

2)  The  events  to  which  end  products  must  respond. 

Define  all  external  stimuli  impinging  on  the  system  that  elicits  a  response. 
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3)  The  physical  and  functional  interfaces  (e.g.,  mechanical,  electrical,  thermal,  data,  and  procedural) 
including  physical  interactions  (e.g.,  form  and  fit),  system  boundaries  (what  is  controlled  by  the 
developer)  and  interactions  (e.g.,  information  flows  and  behaviors)  of  products  or  environments 
within  developer  control  and  those  systems  or  environments  outside  system  boundaries. 

Provide  a  detailed  definition  of  each  external  interface  to  the  system,  typically  documented  in  an 
Information  Exchange  Requirements  (IER),  Interface  Requirements  Document  (IRD)  and  an 
Interface  Control  Document. 

4)  What  system  end  products  must  be  able  to  accomplish  (functional  requirements)  to  satisfy  acquirer 
identified  requirements.  Includes  factors  such  as  producibility,  testability,  transportability, 
installability,  operability,  supportability,  disposability,  reliability,  availability,  maintainability, 
security,  and  safety. 

Functional  requirements  serve  to  translate  operational  needs  into  system  capabilities.  This  is  the 
first  stage  in  a  sequence  of  decompositions  leading  to  design.  The  mission  should  be  examined 
and  characterized  in  measurable  requirement  categories  such  as:  quantity,  quality,  coverage, 
timeliness,  and  availability.  An  example  of  typical  measurables  for  various  systems  is  shown  in 
the  Figure  4.3.1b.  Actual  systems  will  have  many  measurables  under  each  attribute  and  additional 
attributes  such  as  communications,  command  and  control,  security,  etc. 


MEASURABLE 

ATTRIBUTE 

SURVEILLANCE 

SATELLITE 

COMMUNICATION 

SATELLITE 

SUBMARINE 

AIRCRAFT 

QUANTITY 

Frames/Day, 

Sq  Mi/Day 

Throughput  (BPS) 

No.  of  Missiles 
Carried 

Wt.  of  Bombs  or 
Armaments  (lb) 

QUAEITY 

Resolution  (Ft) 

S/N  or  BER 

Targeting  Accuracy 

(ft) 

Navigation 
Accuracy  (ft) 

COVERAGE 

Latitude  &  Long, 
(deg) 

Latitude  &  Long,  (deg) 

Range  (mi) 

Range  (mi) 

TIMELINESS 

Revisit  Time  (hr), 
Proc/Del  Time  (sec) 

Channel  Availability  on 
Demand  (min) 

Time  to  get  on-station 
(hr) 

Time  to  acquire 
target (sec) 

AVAILABILITY 

Launch  Preparation 
Time  (days) 

Bandwidth  Under 
Stressed  Conditions 
(Hz) 

Cruise  Duration 
(days) 

Flight  Prep  Time 
(min) 

Figure  4.3.1b  -  Examples  of  system  attributes  and  measurables 


It  is  important  to  note  that  as  a  result  of  the  system  analysis  and  flowdown,  top-level  functional 
requirements  usually  become  lower  level  performance  requirements.  For  example: 

a.  System  -  Transmit  collected  data  in  real  time  to  remote  ground  site 

b.  Segment  -  Provide  wideband  data  link  from  spacecraft  to  relay 

c.  Element  -  Provide  10  MHz  link  at  17.0  GHz 

d.  Subsystem  -  Provide  10  MHz  link  at  17.0  GHz  with  10  W  effective  radiated  power  for  20 
minutes  maximum  per  orbital  revolution. 

The  top-level  performance  measures  are  used  to  derive  lower-level  subsystem  requirements  for 
configuring  components.  An  example  of  this  would  be  the  conversion  of  the  mission  requirement 
for  aircraft  target  detection  size  and  range  into  dedicated  power,  pulse  width,  and  timing  stability 
which  could  then  be  used  by  the  designer  of  the  radar  system  in  sizing  the  hardware.  As  the  above 
example  illustrates,  the  level  of  detail  to  be  specified  is  driven  by  the  system  level  being 
addressed. 
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The  concept  of  allocation  is  a  useful  technique  to  setting  top-level  technical  requirements,  organizing 
decompositions,  and  controlling  the  subsequent  implementation  to  ensure  compliance.  The  most 
straightforward  application  of  allocation  is  the  direct  apportioning  of  a  value  to  its  contributors.  The 
resulting  allocation  for  a  specific  area,  such  as  pointing  error,  is  usually  referred  to  as  a  budget.  The 
technical  budget  represents  an  apportionment  of  a  performance  parameter  to  several  sources.  This  may 
be  a  top-down  allocation,  such  as  pointing  error  budget,  or  a  bottom-up  summation,  such  as  an 
electrical  power  budget.  Characteristics  such  as  pointing  error  or  electrical  power  distribution  would 
normally  become  parameters  for  Technical  Performance  Measurement  (TPM). 

This  will  eventually  result  in  the  conversion  from  mission  parameters  (targets/sq.  mi.)  into  parameters 
that  the  hardware  and  software  designers  can  relate  to  (Effective  Radiated  Power,  Pointer  Error,  etc.). 
Functional  decomposition  tools  such  as  functional  block  diagrams,  functional  flow  diagrams,  time 
lines,  and  context  diagrams  are  useful  in  developing  requirements.  Quality  Function  Deployment 
(QFD)  is  also  useful,  particularly  where  the  "voice  of  the  customer"  is  not  clear.  As  requirements  are 
derived,  the  analyses  that  led  to  their  definition  must  be  documented  and  placed  into  the  database. 

ENGINEERING  SPECIALTY  REQUIREMENTS: 

Care  must  be  exercised  that  the  myriad  of  engineering  specialty  requirements  and  constraints  are 
incorporated.  Product  Development  Teams  (PDTs)  are  a  way  of  insuring  that  their  requirements  are 
incorporated  into  appropriate  specifications. 

Guidance  recommendations  for  various  technical  specialties  will  vary  depending  upon  the  nature  of  the 
program.  Appendix  G  lists  engineering  specialty  references  specific  to  those  disciplines.  The  IPT  is 
responsible  for  determining  what  technical  support  is  required  to  achieve  the  technical  objectives  of  the 
program. 

•  The  Engineering  Specialty  Table  Appendix  G  highlights  the  more  common  technical  specialties 
and  DoD  source  documents  containing  recommended  procedures.  Those  procedures  should  be 
employed  through  the  tailored  application  of  the  relevant  standards  and  guides,  adapted  to  specific 
program  characteristics. 

•  The  systems  engineering  process  will  allocate  system  requirements  to  establish  clear  technical 
requirements  for  each  technical  specialty  in  a  contract  concurrent  manner  to  support  the  integrated 
system  design.  The  systems  engineering  process  will  collectively  analyze  the  design 
specifications,  conduct  trade-offs,  balance  total  system  requirements,  and  establish  the  final 
configuration. 

c)  Define  performance  requirements  (how  well  each  functional  requirement  must  be  accomplished), 
including  identification  of  key  performance  parameters. 

The  following  are  defined:  (1)  the  performance  expectations  for  each  functional  requirement  (how 
well  the  function  must  be  accomplished);  (2)  the  set  of  Measure  of  Performance  (MOPs)  made  up  of 
the  functional  and  performance  requirement  combinations  associated  with  each  MOE;  (3)  the  Key 
Performance  Parameters  (KPPs)  selected  from  the  MOPs  that  will  be  key  indicators  of  end  product  or 
system  performance,  and  if  not  met,  that  will  cause  the  associated  MOE  to  not  be  satisfied  and  will  put 
the  project  in  cost,  schedule,  or  performance  risk;  and  (4)  functional  and  performance  verification 
approach  for  each  requirement  statement. 

Performance  requirements  shall  be: 

•  derived  based  on  customer  provided  Measures  of  Effectiveness  (MOEs).  When  measures  of 
effectiveness  are  not  provided  at  the  level  of  detail  needed,  the  engineer  shall  develop  and  use  a  set 
of  measures  of  effectiveness  relating  to  customer  missions;  utilization  environment(s);  needs, 
requirements,  and  objectives;  and  design  constraints; 

•  interactively  developed  across  all  identified  functions  based  on  system  life  cycle  factors;  and 

•  characterized  in  terms  of  the  degree  of  certainty  in  their  estimate,  the  degree  of  criticality  to 
system  success,  and  their  relationship  to  other  requirements  (MIL-STD-499B). 
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Typical  performance  parameters  include  range,  accuracy,  response  time,  probability  of  detection,  and 
probability  of  kill.  To  establish  timing-related  performance  requirements,  high-level  function  flows, 
bounded  by  driving  timelines,  are  recommended.  Detailed  Functional  Flow  Block  Diagrams  (FFBDs) 
can  then  be  applied,  as  defined  in  Sub-process  17. 

Finally,  add  information  to  trace  the  function  timing  from  user-defined  performance  requirements  to 
confirm  operational  correctness  or  to  expose  dynamic  inconsistencies.  In  the  latter  case,  record 
inconsistencies  in  the  design  decision  database  to  ensure  eventual  resolution. 

d)  Analyze  acquirer  and  other  stakeholder  requirements,  and  derived  functional  and  performance 
requirements  to  define  human  interface  requirements,  establish  capacities  and  timing,  define 
technology  and  product  design  constraints,  define  enabling  product  requirements,  identify  conflicts, 
and  determine  criteria  for  Trade-off  analyses  to  resolve  conflicts. 

1)  Define  Human  System  Integration  Effects  -  Define  the  operator  roles,  as  applicable,  and  the 
human  interface  requirements  (ergonomic  limitations,  workspace,  eye  movement,  access,  cultural 
background,  natural  and  induced  environmental  constraints,  work  tasks,  and  time  constraints) 
associated  with  functional  and  performance  requirements  on  potential  users,  operators,  installers, 
or  recipients  and  handlers  of  system  end  products. 

Early  inclusion  of  human  interfaces  in  requirements  definition  assures  a  good  user  interface  and  a 
system  that  achieves  the  required  performance  by  operators,  control  and  maintenance  personnel. 
The  Engineering  Specialty  Table  (Appendix  G)  cites  DoD  source  documents  containing 
recommended  procedures. 

2)  Do  the  required  concurrency  capacities  (e.g.,  memory,  storage,  and  flows)  of  end  products  and 
timing  of  events,  states,  modes,  and  functions  related  to  each  operational  profile. 

Ensure  that  concurrent  functions  are  clearly  depicted  in  a  timeline  analysis  covering  the  entire 
system.  A  composite  picture  of  total  demand  on  the  system  (particularly  ‘worst  case’  scenarios)  is 
essential.  Add  traceability  information  to  the  database  to  record  what  external  systems  stimulate 
the  functions,  traced  from  functional  source  requirements. 

3)  Determine  any  constraints  that  will  influence  or  affect  end  product  design  (e.g.,  materials,  special 
skills,  and  automated  tools),  required  physical  characteristics  (e.g.,  size,  color,  texture,  weight,  and 
buoyancy),  operator  safety,  system  security,  reuse  requirements,  standardization  of  end  products, 
open  system  architecture,  maintainer  access,  handling  and  storage,  transportability,  and  other 
attributes  of  end  products  or  design  processes  of  which  trade-offs  cannot  be  made. 

Design  constraints  recognize  inherent  limitations  on  the  sizing  and  capabilities  of  the  system,  its 
interfacing  systems,  and  its  operational  and  physical  environment.  These  typically  include  power, 
weight,  propellant,  data  throughput  rates,  memory,  and  other  resources  within  the  vehicle  or  which 
it  processes.  These  resources  must  be  properly  managed  to  insure  mission  success. 

Design  constraints  are  of  paramount  importance  in  the  development  of  derivative  systems.  A 
derivative  system  is  a  system,  which  by  mandate  must  retain  major  components  of  a  prior  system. 
For  example,  an  aircraft  may  be  modified  to  increase  its  range  while  retaining  its  fuselage  or  some 
other  major  components.  The  constraints  must  be  firmly  established:  Which  components  must 
remain  unmodified?  What  can  be  added?  What  can  be  modified?  The  key  principle  to  be  invoked 
in  the  development  of  derivative  systems  is  that  the  requirements  for  the  system  as  a  whole  must 
be  achieved  while  conforming  to  the  imposed  constraints. 

Within  this  realm  of  system  definition,  Systems  Engineering  personnel  may  also  withhold  a 
margin  to  accommodate  unforeseen  problems.  The  margin  is  held  at  the  system  level.  In 
communication  links,  typically  a  3  dB  system  margin  is  maintained  throughout  the  development 
phase.  These  allocations  are  analyzed  by  Engineering  personnel  to  verify  their  achievability.  As 
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the  design  progresses,  the  current  status  of  the  allocations  is  reviewed  at  the  control  board 
meetings.  Care  must  be  exercised  that  "margins-on-margins"  are  not  overdone,  resulting  in  too 
conservative  (possibly  too  expensive)  a  design. 

The  following  is  a  suggested  approach  for  design  constraint  definitions: 

•  Identify  from  the  input  documents  all  design  constraints  placed  on  the  program.  This 
particularly  includes  compliance  documents,  such  as  previously  approved  specifications  and 
baselines,  standard  end  items,  non-developmental  items,  and  reverse  requirements. 

•  Analyze  the  appropriate  standards  and  lessons  learned  to  derive  requirements  to  be  placed  on 
the  hardware  and  software  Cl  design. 

•  Identify  the  cost  goals  allocated  to  the  design. 

•  Define  system  interfaces  with  other  systems,  human  and  environments,  and  identify  or  resolve 
any  constraints  that  they  impose.  Human  interfaces  include  information  displays  and 
operation  controls.  Environmental  interfaces  include  sensing  devices. 

•  Define  COTS  or  NDI  CIs  constraints  from  those  identified  in  Sub-process  5. 

•  Document  all  derived  requirements  in  specifications  and  insure  that  they  are  flowed  down  to 
the  Cl  level. 

•  Insure  that  all  related  documents  (operating  procedures,  etc.)  observe  the  appropriate 
constraints. 

•  Review  the  design  as  it  evolves  to  insure  compliance  with  documented  constraints. 

4)  Define  technical  requirements  for  enabling  products  associated  with  processes  to  develop, 
produce,  test,  deploy/install,  operate,  support/maintain,  train,  and  retire/dispose  of  end  products 
under  development  or  being  improved.  Identify  and  resolve  requirements  that  have  questionable 
utility  or  have  unacceptable  risk  of  not  being  satisfied. 

The  above  analysis  is  usually  directed  at  the  mission  or  payload  requirements  and  does  not 
consider  the  total  system  requirements,  which  include  communications,  command  and  control, 
security,  supportability,  life  expectancy,  etc.  It  is  necessary  to  expand  the  analysis  to  include 
supporting  areas  in  order  to  obtain  the  total  system  requirements. 

5)  Identify  conflicts  among  the  requirements  set. 

Identify  all  user  requirements,  which  lead  to  conflicting  technical  requirements.  These  frequently 
arise  when  the  performance  in  one  area  adversely  affects  performance  in  another. 

6)  Define  the  set  of  risk,  cost,  schedule,  and  performance  criteria  to  be  used  in  conducting  trade-off 
analyses  for  conflict  resolution. 


NOTES 

1  Developers  are  to  ensure  that  residual  risks  from  constraints  are  not  significant  to  harm  or 
otherwise  prevent  the  system  from  performing  its  functions,  create  unacceptable  costs,  or  price  the 
system’s  end  products  out  of  competitiveness. 

2  Analyses  of  system  requirements  can  necessitate  consideration  of  existing  or  possible  physical 
solutions  to  ensure  feasibility. 


Cost  trade  studies  are  initiated  in  order  to  identify  cost  "drivers"  or  areas  where  resources  can  best  be 
applied  to  achieve  the  maximum  cost  benefit.  These  studies  should  examine  those  performance 
parameters  where  small  changes  in  the  parameters  produce  significant  changes  in  costs  or  risks, 
commonly  known  as  cost  sensitivity  analysis.  For  example,  sometimes  a  relatively  small  change  in 
mean-time-to-repair  (MTTR)  or  mean-time-between-failures  (MTBF)  results  in  large  savings  in 
operational  costs.  Significant  cost  and  risk  drivers,  once  identified,  can  greatly  assist  requirements 
conflict  resolution.  These  studies  also  help  to  identify  areas  in  which  emphasis  can  be  placed  during 
the  subsequent  sub  phases  to  obtain  the  maximum  cost  reduction. 
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e)  Identify  and  resolve  requirements  that  have  questionable  utility  or  have  unacceptable  risk  of  not 
being  satisfied. 

Examine  any  adverse  consequences  of  incorporating  requirements. 

•  Is  unnecessary  risk  being  introduced? 

•  Is  the  system  cost  within  budget  limitations? 

•  Is  the  technology  ready  for  production? 

•  Are  sufficient  resources  available  for  production  and  operation? 

•  Is  the  schedule  realistic  and  achievable? 

f)  Resolve  identified  conflicts  between  the  requirements,  e.g.,  sets  of  acquirer  requirements  and  other 
stakeholder  requirements,  and  among  these  sets  (see  Sub-process  23). 

The  systems  engineer  does  not  perform  mission  analysis  and  requirements  analysis  as  discrete 
sequential  operations.  Rather  the  analyses  are  performed  concurrently  with  mission  needs  playing  the 
dominant  role.  It  is  essential  that  the  system  engineer  proceed  in  this  manner  to  assure  progression 
toward  the  most  cost-effective  solution  to  the  mission  need.  Throughout  this  process,  the  systems 
engineer  makes  cost/requirements  trade-offs.  The  significant  or  controversial  ones  are  formally 
documented  and  presented  to  the  customer  for  review.  Following  mission/requirements  analysis, 
system  functional  analysis  proceeds  leading  to  candidate  system  design(s),  which  are  evaluated  in 
terms  of  performance,  cost,  and  schedule.  While  this  process  ideally  results  in  an  optimum  technical 
system,  in  actuality,  limitations  on  cost,  schedule,  and  risk  place  constraints  on  system  design  which 
result  in  selection  of  a  preferred  system  from  a  number  of  candidates,  rather  than  the  optimum 
technical  solution. 

Where  existing  user  requirements  cannot  be  confirmed,  trade  studies  should  be  performed  to  determine 
more  appropriate  requirements  to  achieve  the  best-balanced  performance  at  minimum  cost.  Where 
critical  resources  (Weight,  Power,  Memory,  Throughput,  etc.)  must  be  allocated,  trade  studies  may  be 
required  to  determine  the  proper  allocation. 

g)  Prepare  a  set  of  system  technical  requirement  statements  that  are  well  formulated  in  accordance 
with  Sub-process  25. 

Assess  requirements  as  to  degree  of  certainty  of  estimate,  and  place  a  “To  Be  Reviewed”  (TBR)  flag 
after  any  requirement  that  is  not  completely  agreed  upon,  or  a  “To  Be  Determined”  (TBD)  flag  where 
the  value  is  unknown.  Place  a  list  of  all  TBD/TBR  items  with  responsibilities  and  closure  dates  at  the 
back  of  the  specification. 

Prioritize  all  requirements  as  to  the  criticality  of  mission  success.  Since  resources  on  any  program  are 
limited,  this  identifies  where  the  effort  should  be  concentrated  in  refining,  deriving,  and  flowing  down 
requirements. 

h)  Ensure  that  the  set  of  system  technical  requirements  is  correct  in  accordance  with  Sub-process  28. 

The  system  technical  requirements  are  documented  in  a  System  Requirements  Document  (SRD), 
which  is  validated  in  accordance  with  Sub-process  28. 

i)  Record  the  resulting  set  of  system  technical  requirements  in  the  established  information 
database. 

The  validated  set  of  system  technical  requirements  and  associated  assumptions  is  captured  in  the  project’s 
information  database  and  maintained  and  controlled  throughout  the  life  of  the  project  in  accordance  with  the 
Outcomes  Management  Sub-process  12. 

NOTE  -  Controlled  maintenance  of  the  system  technical  requirements  in  the  information  database  allows  for 
traceability,  supports  validation,  and  is  essential  for  change  management. 
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Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Utilization  environment  (SP  1 6) 

•  Verification  approach  (SP  16) 

•  Operational  profiles  (SP  16,  17) 

•  Physical  and  functional  requirements  (SP  16,  17) 

•  Mission  Profiles  (SP  16,  17) 

•  Cycle  timelines  (SP  16,  17) 

•  Measures  of  Performance  (MOP)  (SP  16,  17) 

•  Key  Performance  Parameter  (KPP)  (SP  10,  16,  17) 

•  Functional  performance  (SP  16,  17) 

•  Human  interface  requirements  (SP  16,  17) 

•  Function  concurrency  /  capacity  (SP  16,  17) 

•  Technology  constraints  (SP  16,  18) 

•  Design  constraints  (SP  16,  18) 

•  Enabling  products  requirements  (SP  16,  17) 

•  Conflicting  requirements  (SP  16,  17) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Trade  Options  and  Constraints  (SP  23) 

•  System  Requirements  Document  (SRD)  (SP  11,  17) 

•  System  technical  requirements  (SP  5,  6,  8,  11,  17,  25,  28,  30) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  6:  Schedule  and  Organization 
Sub-process  8:  Work  Directives 
Assessment  Process 

Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 
Sub-process  28:  System  Technical  Requirements  Validation 
System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 

Agents 

Logistics,  Ops  Analysis,  Systems  Engineering,  Test,  Specialty  Engineering,  User 
Tools 

Functional  Flow  Block  Diagram  (FFBD),  Quality  Function  Deployment  (QFD),  Context  Diagram,  Timeline 
Analysis 
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References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
IEEE  1220 

MIL-STD-499B 

System/Subsystem  Specification  (SSS)  Data  Item  Description  (DI-IPSC-81431) 

World  Class  Example,  Jerry  Lake,  1999. 

Metrics  and  Measures 

Percent  completion  of  analysis  and  output  products. 

Percent  of  system  technical  requirements  that  have  been  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  a  set  of  system  technical  requirements  that  are  unambiguous, 
complete,  consistent,  achievable,  verifiable,  and  necessary  and  sufficient  for  a  system  design. 
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4.3.2  Solution  Definition  Process 


The  Solution  Definition  Process  is  used  to  generate  an  acceptable  design  solution.  This  solution  satisfies:  (1) 
the  system  technical  requirements  resulting  from  completing  the  Requirements  Definition  Process  described  in 
Subsection  4.3.1;  and  (2)  the  derived  technical  requirements  from  the  Solution  Definition  Process  described  in 
this  subsection.  The  relationships  of  the  Solution  Definition  Process/Sub-processes  are  shown  in  Figure  4.3.2a. 
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Figure  4.3.2a  -  Solution  Definition  Process/Sub-process  relationships 
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The  three  sub-processes  associated  with  the  Solution  Definition  Process  are  shown  in  Figure  4.3.2b. 


Sub-process  17  -  Logical  Solution  Representations 


Solution 

Definition 

Process 

Requirements 


Sub-process  18  -  Physical  Solution  Representations 


Sub-process  19  -  Specified  Requirements 


Figure  4.3.2b  -  Solution  Definition  Process/Sub-processes 


NOTE  -  The  purpose  of  the  sub-processes  related  to  the  Solution  Definition  Process  is  to  solve  the  technical 
problem.  This  involves  identifying  alternative  end  products  for  the  system,  selecting  and  defining  an  optimal 
set  of  end  products,  defining  the  feasible  subsystems  related  to  the  end  products,  identifying  requirements  for 
enabling  products,  and  identifying  needed  high-risk  technology  developments. 


Sub-process  17-  Logical  Solution  Representations 

The  developer  shall  define  one  or  more  validated  sets  of  logical  solution  representations  that 
conform  with  the  technical  requirements  of  the  system. 


Preceding  Process 
Requirements  Design  Process 

Sub-process  16:  System  Technical  Requirements 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 
Requirements  Validation  Process 
Sub-process  25:  Requirements  Statements  Validation 

Sub-process  29:  Logical  Solution  Representations  Validation 

Inputs 

•  System  Technical  Requirements  (SP  16) 

•  Operational  Capabilities  (SP  1 6) 

•  Physical  and  functional  requirements  (SP  16) 

•  Mission  Areas  (SP  16) 

•  Cycle  timelines  (SP  16) 

•  Measures  of  Performance  (MOP)  (SP  1 6) 

•  Key  Performance  Parameter  (KPP)  (SP  1 6) 

•  Functional  performance  (SP  16) 

•  Human  interface  requirements  (SP  1 6) 

•  Function  concurrency  /  capacity  (SP  16) 

•  Enabling  products  requirements  (SP  1 6) 

•  Conflicting  requirements  (SP  1 6) 

•  System  Requirements  Document  (SRD)  (SP  1 6) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Trade-off  Analysis  Technical  Report  (SP  23) 

•  Requirement  statement  validation  revisions  (SP  25) 

•  Logical  solution  representation  validation  revisions  (SP  29) 
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Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 


Tasks 

The  reason  for  developing  logical  solutions/functional  system  representations  is  to  define  Derived  Technical 
Requirements  (DTR).  Identified  logical  representations  shall  be  analyzed  to  determine  the  lower  level 
requirements  to  accomplish  the  parent  requirements.  All  specified  usage  modes  shall  be  analyzed.  Logical 
solution  requirements  shall  be  arranged  so  that  lower  level  requirements  (derived  or  otherwise)  are  recognized 
as  part  of  higher  level  requirements  (assure  traceability  from  output  products  from  Initial  Specification  from 
Acquirer,  Sub-process  14,  Other  Requirements  from  Internal  and  External  Sources,  Sub-process  15  and  System 
Technical  Requirements,  Sub-process  16)  (see  Chapter  4.2.3  of  this  Guidebook  for  reference  to  the 
Requirements  Traceability  Matrix).  For  example  the  logical  solution  representation  should  be  traceable  to  the 
functional  description  and  functional  flow  block  diagram. 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 


following: 


a)  Select  and  implement  one  or  more  appropriate  approaches  to  providing  an  abstract  definition  of 
the  solution  to  the  system  technical  requirements.  For  the  approaches  selected,  complete  the 
appropriate  tasks  from  b)  through  d)  below  that  aid  in  defining  logical  solution  representations. 

The  approach  can  be  a  combination  of  various  approaches  tailored  to  the  type  of  system  at  a  given 
system  level.  The  application  of  the  various  analyses,  or  a  combination  thereof,  is  dependent  on  many 
variables,  such  as  system  type  (e.g.,  hardware,  or  software),  size,  and  the  functional  complexity. 

The  traditional  systems  engineering  approach  for  developing  Logical  Solution  Representations  has 
been  the  Functional  Analysis.  This  approach  is  primarily  supported  by  the  development  of  Functional 
Flow  Block  Diagrams  and  the  Functional  Decomposition  methods.  Other  types  of  analyses  have  been 
developed  to  support  Logical  Solution  Representations;  each  method  favors  particular  system  types 
and  development  activities  and  has  advantages  and  disadvantages.  For  example,  the  Structured 
Analysis ,  which  includes  context  diagrams,  control/data  flows,  data  dictionaries,  entity-relationships 
diagrams,  and  state  transition  diagrams,  is  typically  applied  in  development  of  complex  software 
intensive  systems  (i.e.,  Air  Traffic  Control  System).  Another  type,  the  Object  Oriented  Analysis  using 
Use  Case  /Unified  Modeling  Language  (UML),  is  commonly  applied  in  the  development  of 
information  systems  and  other  software  applications.  The  resultant  output  of  this  task  is  typically  a 
logical  solution  analysis  approach.  The  analyses  considered  for  the  range  of  system/software  are 
shown  in  Figure  4.3.2c.  In  task  b),  one  must  establish  a  method  /  approach  to  the  System  Technical 
Requirements  (STR).  That  task  defines  these  methods  in  more  detail  including  the  specific  procedures 
that  should  be  considered  for  developing  a  Logical  Solution. 

A  combination  of  these  may  be  used  for  a  system  that  contains  both  hardware  and  software.  One 
approach  might  be  to  perform  a  functional  analysis  at  the  system  level  and  use  Object  Oriented 
Analysis  (00 A)  for  the  software  elements.  If  multiple  approaches  are  used,  traceability  must  be 
maintained  across  methodologies. 

Analyses: 


Functional 


Structured 


Object  Oriented 


◄ 


Complexity/Functionality 


► 


Simple  system 
Little  software 


Complex  system 
Software  Intensive 


Figure  4.3.2c  -  Analyses  considered  for  system/software 
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NOTE  -  Functional  analysis,  object-oriented  analysis,  structured  analysis,  and  information 
engineering  analysis  are  recognized  approaches  found  in  text  books  and  other  literature  to  develop 
logical  solution  representations  in  terms  of,  for  example,  functional  flows,  behavioral  responses, 
state  and  mode  transitions,  timelines,  control  flows,  data  flows,  information  models,  object 
services  and  attributes,  context  diagrams,  threads,  data  structures,  and  functional  failure  modes  and 
effects. 


b)  Establish  sets  of  logical  solution  representations  by:  (1)  doing  trade-off  analyses  (see  Sub-process 

23);  (2)  identifying  and  defining  interfaces,  states  and  modes,  timelines,  and  data  and  control  flows;  (3) 

analyzing  behaviors;  and  (4)  analyzing  failure  modes  and  defining  failure  effects. 

Functional  Analysis 

•  Functional  Flow  Block  Diagram  (FFBD):  The  translation  of  the  system  operational  concept  into  a 
series  of  time- sequenced  blocks  that  contain  a  description  of  the  system  function. 

•  Functional  Decomposition:  The  break  down  of  the  system  functions  from  higher  level  to  lower 
level.  This  approach  is  not  time  sequenced. 

•  Timelines  and  Sequencing  -  When  time  is  critical  to  the  sequencing  of  events  that  a  system  must 
perform,  a  time-line  analysis  shall  be  conducted.  A  method  for  defining  timing  and  sequencing  is 
the  Time  Analysis  Sheet  (see  DAU  SE  Fundamentals)  and  Time  Line  Analysis  Chart  (see 
INCOSE  1998,  4.3).  Some  of  the  automated  systems  engineering  tools  provide  the  capability  to 
perform  a  simulation  and  give  time  line  charts. 

Structured  Analysis 

•  Context  Diagram:  A  diagram  that  shows  the  system  and  its  interfaces  with  external 
components/elements . 

•  Control  Data  Flow  Diagrams:  Data  &  Control  Flow  diagrams  are  used  to  document  all  data 
transmission,  control,  and  processing  functional  requirements  (see  INCOSE  1998,  4.3). 

•  Data  Dictionaries:  A  data  dictionary  is  an  organized  listing  of  all  the  data  elements  that  are 
pertinent  to  a  system.  It  should  be  used  to  describe  data  elements  in  both  the  Control  Data  Flow 
Diagrams  and  Context  Diagrams.  It  should  contain  name,  type,  kind,  and  description. 

•  Activity  Models  :  A  diagram  that  identifies  the  system  entities  (other  systems,  devices,  or  people 
that  the  system  must  keep  track  of)  connected  by  an  arrow  that  is  labeled  with  the  cause/effect 
relationship  (verbs)  with  other  entities  in  the  diagram. 

•  State  Transition  Diagrams:  A  diagram  that  shows  the  possible  modes  and  states  that  the  can  exist 
in  the  system  and  the  event  or  action  under  which  the  system  can  transition.  Preliminary  States 
and  Modes  are  derived  from  the  Concept  of  Operations  (Sub-process  14)  and  the  System 
Technical  Requirements  (STRs)  (Sub-process  16)  are  further  refined  in  increased  detail.  A  top- 
level  draft  of  this  may  be  generated  as  a  part  of  Sub-process  16,  task  a. 

Object  Oriented  Analysis  (OOA)  (Booch  1994,  p  155) 

•  Classical  Approach:  Definition  of  the  system  through  categorization  of  things,  roles,  events,  and 
interaction. 

•  Behavior  Analysis:  Definition  of  the  systems  through  the  grouping  of  objects  that  exhibit  similar 
behavior. 
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•  Domain  Analysis:  Definition  of  the  systems  based  on  objects,  operations  and  relationships  that  are 
important  to  the  domain  (technical  area). 

•  Use  Case  Analysis/Unified  Modeling  Language  (UML):  Definition  of  the  system  based  on  a 
particular  form  or  example  of  usage/scenario.  This  also  supports  analyzing  behaviors. 

Logical  Solution  Trade-Off  Analyses 

An  optimum  logical  solution  representation  should  be  developed  by  formulating  alternative  sets  and 
down-selecting  through  the  trade-off  process.  Trade  Studies  (see  Sub-process  23)  of  alternative 
system  logical  solutions  must  be  performed  by  taking  into  account  cost,  customer/user  requirements 
(fleet  project  team  input),  open  system  considerations,  and  constraints  such  as  the  customer  requesting 
the  use  of  a  specific  Commercial  Off-the-Shelf  (COTS)  product  or  interface  with  legacy  systems. 

After  the  appropriate  approach  is  selected  {Functional  Analysis ,  Structured  Analysis ,  or  Object- 
Oriented  Analysis),  ensure  the  following  analytical  techniques  are  applied  in  the  trade-off  decision 
process  where  appropriate. 

•  Defining  Interfaces  (N2  Charts)  -  Logical  solution  requirements  shall  be  sequenced  with  input, 
output,  and  logical  solution  interface  (internal  and  external)  requirements  defined;  and  be  traceable 
from  beginning  to  end  conditions  and  across  their  interfaces.  A  method  for  defining  functional 
interfaces  is  the  N2  chart  (INCOSE  1998,  4.3).  Description  of  interface  is  critical  in  taking  an 
Open  Systems  Approach  to  system  definition. 

•  Analyzing  Behaviors  -  Analyze  system  logical  solution  behavior  through  simulation.  Some  of  the 
automated  systems  engineering  tools  provide  the  capability  to  perform  a  run-time  simulation  and 
check  various  system  logic  and  threads  /  paths  through  the  system  logical  solution  definition. 

•  Failure  Modes,  Effects  and  Criticality  Analysis  (FMEA/FMECA)  -  Analyze,  define  and  prioritize 
logical  solution  (functional  level)  failure  modes  and  effects  through  a  Failure  Modes  and  Effects 
Analysis  /  Failure  Modes  Effects  and  Criticality  Analysis  (FMEA/FMECA)  (see  references  MIL- 
STD-1629  and  DI-ILSS-81 163 A).  This  analysis  shall  be  used  to  define  fault  detection,  isolation, 
and  recovery  functions  such  as  Built-in-Test  and  redundancy  requirements. 

c)  Assign  (i.e.,  perform  Requirements  Allocation  of)  system  technical  requirements  (especially 

performance  requirements  and  constraints  from  the  system  technical  requirements)  to  elements  of  the 
logical  solution  representations,  e.g.,  subfunctions,  groups  of  subfunctions,  objects,  and  data 
structures. 

Establish  performance  requirements  for  each  logical  solution  requirement  (Functional  Area)  and 
interface.  A  method  for  gathering  requirements  allocation  is  the  Requirement  Allocation  Sheet  (RAS) 
(see,  DAU  SE  Fundamentals  and  DI-GDRQ-81222).  Time  requirements  that  are  prerequisite  for  a 
logical  solution  or  set  of  logical  solutions  shall  be  determined  and  allocated.  The  resulting  set  of 
requirements  shall  be  defined  in  measurable  terms,  applicable  go/no-go  criteria,  and  in  sufficient  detail 
for  use  as  design  criteria.  Performance  requirements  shall  be  traceable  throughout  the  logical  solution 
architecture,  through  the  analysis  by  which  they  were  allocated,  to  the  higher-level  requirements  they 
are  intended  to  fulfill.  Logical  solution  architecture  refers  to  logical  solution  definition  of  the  system 
and  the  allocation  of  performance  requirements  to  these  functions,  not  the  hardware/software 
architecture. 
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NOTES 


1  There  can  also  be  system  technical  requirements  that  are  neither  appropriate  to  assign  to 
the  sets  of  logical  solution  representations  nor  modifiable  into  derived  technical 
requirements.  An  example  is  a  characteristic  or  constraint  applicable  only  to  the  system,  not 
to  the  products  of  the  system.  These  system  technical  requirements  must  be  analyzed  and 
assigned  during  Physical  Solution  Representation,  Sub-process  18,  tasks  a),  b),  and  c). 

2  There  will  be  additional  derived  technical  requirements  prepared  to  reflect  system 
analysis  results  from  Physical  Solution  Representation,  Sub-process  18  task  c). 

d)  Identify  and  define  derived  technical  requirement  statements  resulting  from  tasks  a)  and  b). 

Ensure  that  the  derived  technical  requirements  are  stated  acceptably  in  accordance  with  Requirements 
Statements  Validation,  Sub-process  25. 

e)  Ensure  that  each  set  of  logical  solution  representations  is  correct  in  accordance  with  Logical 
Solution  Representations  Validation,  Sub-process  29. 

f)  Record  the  resulting  sets  of  logical  solution  representations,  the  set  of  derived  technical 
requirement  statements,  and  any  unassigned  system  technical  requirements  (see  notes  under  task 
c)  above),  along  with  source  rationale  and  assumptions  in  the  established  information  database  in 

accordance  with  Outcomes  Management  Sub-process  12. 

Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Functional  Analysis  Products  (SP  18) 

-  FFBD  /  Functional  Decomposition 

-  Timeline 

•  Structured  Analysis  Products  (SP  18) 

-  Context  /  QFD  /  Data  Dictionaries  /  Entity-Relationship  /  Modes  &  States  Diagrams 

•  Object  Oriented  Analysis  Products  (SP  18) 

-  Classical  /  Behavior  /  Domain  /  Use  Case  Analyses 

•  N2  /  FMEA  /  FMECA  /  RAS  (SP  1 8) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Trade  options  and  constraints  (SP  23) 

•  Derived  Technical  Requirements  (SP  25) 

•  Logical  Solution  Representation  (SP  18,  29) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 
Sub-process  29:  Logical  Solution  Representations  Validation 

Agents 

Systems  Engineering,  R&M,  Human  Systems  Integration,  Safety,  Design,  Logistics,  Test,  Software 
Development 
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Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™),  Simulations, 
RAS 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
Systems  Engineering  Analysis  (Blanchard) 

Standard  Practice  for  Performing  FMECA  (MIL-STD-1629), 

DI-ILSS-81 163A 

Requirement  Allocation  Sheets  (RAS)  Data  Item  Description  DI-GDRO-81222 
Object  Oriented  Analysis  &  Design  (Booch) 

Modern  Structured  Analysis  (Yourdon) 

Metrics  and  Measures 

Percent  Completion  of  Logical  Solution  Products 

Percent  of  Logical  Solution  Products  that  have  been  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  requirement,  when  combined  with  the  system  technical  requirements,  provide  the  basis  for 
developing  alternative  physical  solution  representations. 


NOTES 

1  Conditions  for  logical  groupings  are  determined  by  many  factors  and  vary  from  one  project  to 
another.  One  common  driver  for  logical  groupings  is  to  enable  the  use  of  existing  products,  and  thus 
lessen  development  time  and  cost.  Another  common  reason  is  to  gain  some  advantage  by  introducing  a 
particular  new  technology.  In  either  of  these  cases,  the  grouping  can  result  in  interfaces  that  did  not 
previously  exist.  New  requirements  have  to  be  derived  to  accommodate  these. 

2  Accomplishment  of  the  tasks  associated  with  this  sub-process  is  often  iterative  because  outcomes 
raise  questions  that  require  certain  tasks  of  the  Requirements  Definition  Process  to  be  reaccomplished. 
In  turn,  certain  tasks  associated  with  defining  logical  solution  representations  and  derived  technical 
requirements  are  reaccomplished.  Such  iteration  is  important  in  order  to  lessen  the  possibility  of  more 
costly  iterations  of  System  Design  Processes  during  a  later  engineering  life-cycle  phase. 


Derived  technical  requirements  and  requirements  associated  with  logical  solution  representations  should  be 
incorporated  into  traceability  procedures.  This  will  enable  ensuring  that  system  technical  requirements  are 
properly  supported  by  the  derived  technical  requirements  and  logical  solution  representations. 


Sub-process  18  -  Physical  Solution  Representations 

The  developer  shall  define  a  preferred  set  of  physical  solution  representations  that  agrees 
with  the  assigned  logical  solution  representations,  derived  technical  requirements,  and  system 
technical  requirements. 


Preceding  Process 
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Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 
Sub-process  24:  Risk  Analysis 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 

Inputs 

•  Design  constraints  (SP  1 6) 

•  Technology  constraints  (SP  16) 

•  Functional  Analysis  Products  (SP  17) 

-  FFBD  /  Functional  Decomposition 

-  Timeline 

•  Structured  Analysis  Products  (SP  17) 

-  Context  /  QFD  /  Data  Dictionaries  /  Entity-Relationship  /  M&S  Diagrams 

•  Object  Oriented  Analysis  Products  (SP  17) 

-  Classical  /  Behavior  /  Domain  /  Use  Case  Analyses 

•  N2  /  FMEA  /  FMECA  /  RAS  (SP  1 7) 

•  Logical  Solution  Representation  (SP  17) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Trade-off  Analysis  Technical  Report  (SP  23) 

•  Risk  Analysis  Report  (SP  24) 

•  Requirement  statements  validation  revisions  (SP  25) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  These  tasks  for  this  sub¬ 
process  are  normally  referred  as  System  Architecture  Synthesis  (see  INCOSE  1998,  Section  4.4).  The  System 
Architecture  Synthesis  is  part  of  the  overall  system  design  process,  and  it  runs  iteratively  with  Requirements 
Definition  and  Functional  Analysis  (Logical  Solution  Representations).  Tasks  to  consider  include  the 
following: 

a)  Analyze  logical  solution  representations,  derived  technical  requirements,  and  any  unassigned 
system  technical  requirements  [see  note  under  Sub-process  17,  task  c)]  to  determine  which  ones  (1) 
provide  requirements  for  enabling  products;  (2)  can  be  done  best  manually  or  by  facilities,  materials, 
data,  services,  or  techniques;  and  (3)  can  be  done  best  by  hardware,  software,  or  firmware  products 
(new  or  existing). 

The  developer  shall  initiate  the  physical  solution  representation  analysis  by  defining  alternatives  of  the 
system  hierarchy.  This  hierarchy  is  described  in  Section  6.2  of  this  document.  These  system  hierarchy 
alternatives  create  the  design  space  for  all  possible  choices  of  elements.  The  system  hierarchy  is 
derived  from  the  logical  solution  representation,  and  its  purpose  is  to  create  the  system  elements, 
which  constitutes  the  building  blocks  from  which  the  system  architecture  is  generated.  The  system 
elements  include  hardware,  software,  information,  procedures,  and  people;  and  are  defined  top  down 
beginning  with  the  system,  subsystem,  and  configuration  items. 
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NOTES  -  The  system  hierarchy  can  be  applied  in  the  planning  process  to  develop  the  Work 
Breakdown  Structure  (WBS)  in  accordance  with  the  building  block  concept  that  consists  of  the 
breakdown  of  end  products  and  enabling  products. 


b)  Assign  representations  from  Sub -process  17  unassigned  system  technical  requirements,  and 
derived  technical  requirements  to  physical  entities  that  will  make  up  a  physical  solution. 

The  developer  shall  assign  (Requirements  Allocation)  logical  solution  representation  in  the  form  of 
functions  and  system  technical  requirements  (i.e.,  performance,  reliability,  maintainability,  interfaces, 
environmental  requirements,  human  systems  integration,  survivability,  safety,  security,  supportability, 
materials,  cost,  and  other  constraints)  to  the  physical  elements  in  the  system  hierarchy,  thus  creating  a 
design  space  and  range  of  values  for  those  physical  elements  alternatives.  These  allocations  and 
design  descriptions  for  each  physical  element  should  not  be  constrained  by  the  values  of  other 
elements.  Assignments  (Allocation)  of  design  requirements  shall  be  based  on  the  mathematical 
formulation  and  representations  relative  to  that  discipline  (i.e.,  Performance  Models,  Reliability  & 
Maintainability  Model  and  Schema,  etc.).  After  requirements  assignments  are  completed,  the  next  step 
is  the  identification  of  the  Systems  Hierarchy  Specification  Tree  for  the  various  system  elements 
alternatives. 


NOTES  -  The  assignment  to  physical  entities  and  the  generation  of  alternative  solutions 
composed  of  these  entities  are  tightly  coupled  and  iterative. 


c)  Generate  alternative  physical  solutions  by: 

Sizing,  configuring,  and  integrating  of  the  physical  system  elements  alternatives  in  relation  to  the 
logical  representation  options  and  assigned  requirements  range.  At  this  point,  the  developer  shall 
begin  to  synthesize  the  system  architecture  alternatives.  This  approach  together  with  the  Schematic 
Block  Diagram  (SBD),  Systems  View  (C4ISR  Architecture  Framework)  and  N2  diagrams  enables  the 
generation  of  architectural  alternatives  (see  DAU  SE  Fundamentals,  Chapter  6  and  INCOSE  1998 
Section  4.4.3  for  further  details).  In  developing  these  architectural  alternatives,  the  developer  shall 
consider  the  following: 

1)  Identification  and  definition  of  physical  interfaces  to  include  Information  Exchange 
Requirments  (IERs) 

2)  Identification  and  analysis  of  critical  parameters  (MOEs  and  TPMs) 

3)  Identification  and  assessment  of  physical  solution  options: 

a.  Technology  Requirements 

b.  Off-the-shelf  availability  and  non-developmental  items  (NDI) 

c.  Competitive  considerations 

d.  Failure  modes,  effects,  and  criticality  (Integrated  Diagnostics  /  Testability) 

e.  Performance  assessment 

f.  Life  cycle  considerations 

g.  Capacity  to  evolve 

h.  Make  versus  buy 

i.  Standardization  considerations  (Open  System  Architecture) 

j .  Integration  concerns 

4)  Performance  of  system  analysis  (see  Sub-process  22,  23,  and  24),  including  performance  design 
and  parametric  analyses  to  optimize  operating  target  parameters.  This  effort  helps  establish 
sensitivities,  connects  hardware  requirements  to  mission  measurables,  exposes  thresholds  and 
risks,  and  creates  the  range  for  robust  design  goals.  The  System  Analysis  will  include 
considerations  in  the  design  for:  performance,  cost,  reliability  and  maintainability,  testability 
(reference  integrated  diagnostics,  supportability,  manufacturability,  maintainability,  safety, 
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security,  and  producibility).  Supportability  and  Logistics  Support  Analysis  (LSA)  plays  a  key  role 
in  the  development  of  physical  solution  representation.  For  many  of  the  above  “ilities”,  the  Navy 
has  specific  functional  divisions  within  the  Systems  Engineering  department.  Appendix  G  lists 
many  of  the  references  for  these  disciplines.  This  should  include  analyses  of  R&MSS,  Human 
Systems  Integration  Engineering,  Electromagnetics  (EM),  Survivability,  Materials,  Parts, 
Environmental,  Supportability  Design,  LSA,  Open  System,  COTS/NDI,  and  System  & 
Performance  Design. 

d)  Identify  and  define  derived  technical  requirement  statements  resulting  from  tasks  a),  b),  and  c) 
that  are  stated  acceptably  in  accordance  with  Sub-process  25  (see  INCOSE  1998,  Section  4.4.4  for 
further  details). 

e)  Select  the  preferred  physical  solution  representation  for  further  characterization  into  a  design 
solution  from  the  evaluation  of  each  physical  solution  representation  results  (see  Sub-process  22,  23, 
and  24).  Document  the  physical  solution  concept  using  the  Concept  Description  Sheet  (see  DAU  SE 
Fundamentals)  and  the  Design  Sheet. 

f)  Ensure  that  the  selected  physical  solution  representation  is  consistent  with  the  assigned  logical 
solution  representations,  derived  technical  requirements,  and  any  unassigned  system  technical 
requirements  (see  note  under  Sub-process  17,  task  c). 

g)  Record  the  selected  physical  solution  representation  and  the  outcomes  of  task  d)  above,  along  with 
selection  rationale  and  assumptions,  in  the  established  information  database. 

Derived  technical  requirements  and  requirements  associated  with  logical  solution  representations  should  be 
incorporated  into  traceability  procedures.  This  will  ensure  that  system  technical  requirements  are  properly 
supported  by  the  derived  technical  requirements  and  logical  solution  representations. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Effectiveness  Analysis  Request  (for  alternative  physical  solutions)  (SP  22) 

•  Trade  options  and  constraints  (SP  23) 

•  Risk  Analysis  Request  (SP  24) 

•  Physical  Solution  Options  (SP  18) 

•  Derived  technical  requirements  (SP  25) 

•  Selected  physical  solution  representation  (SP  19,  30) 

(to  include  supporting  documentation,  e.g.,  Concept  Description  Sheet,  Design  Sheet,  System  Hierarchy 
Definition,  Functional  and  Performance  Allocation,  System  Specification  Tree  (HWCI  /  CSCI),  FFBD  & 
System  Schematic,  FMEA  /  FMECA  (Based  on  FFBD),  Integrated  Diagnostic  Analysis  (Testability), 
System  Architecture  Views) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 
Sub-process  24:  Risk  Analysis 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 
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System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 

Agents 

Systems  Engineering,  R&M,  Human  Systems  Integration,  Safety,  Security,  Design,  Logistics,  Test, 
Producibility,  Software  Design 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

SBD 

N2  Diagrams 

Requirement  Allocation  Sheets  (RAS) 

Concept  Description  Sheet  and  Design  Sheet 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
Systems  Engineering  Analysis  (Blanchard) 

Standard  Practice  for  Performing  FMECA  (MIL-STD-1629) 

Metrics  and  Measures 

•  Percent  Completion  of  Physical  Solution  Products 

•  Percent  of  Physical  Solution  Products  that  have  been  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  the  preferred  physical  solution  representation  that  will  be  fully 
characterized  during  Sub-process  19.  Additionally  the  outcomes  show  that: 

(1)  The  preferred  physical  solution  representation  satisfies  the  assigned  requirements  of  the  logical  solution 
representations,  derived  technical  requirements,  and  system  technical  requirements;  and 

(2)  The  preferred  physical  solution  representation  is  upward-  and  downward-traceable  with  respect  to  the 
assigned  requirements  of  logical  solution  representations,  derived  technical  requirements,  and  any 
unassigned  system  technical  requirements  [see  notes  under  Sub-process  17,  task  c)]. 

Outcomes  can  be  displayed  as  a  hierarchical  structure  of  physical  entities,  schematics,  physical  models, 
analytical  models,  or  explosion  diagrams. 


NOTES 

1  As  each  physical  solution  representation  is  defined,  it  usually  is  necessary  to 
reaccomplish  tasks  related  to  the  definition  of  logical  solution  representations  to  ensure  that 
the  final  set  of  derived  requirements  and  requirements  associated  with  logical  solution 
representation  is  traceable  to  the  preferred  physical  solution  representation,  and  vice  versa. 

2  Physical  solution  representation  will  eventually  be  composed  of  one  or  more  of  the 
following:  hardware,  software,  firmware,  material,  data  (e.g.,  manuals,  and  handbooks), 
doctrine,  organization,  training,  materiel,  leadership,  personnel  and  facilities  (DOTMLPF). 


Sub-process  19  -  Specified  Requirements 

The  developer  shall  specify  requirements  for  the  design  solution. 
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Preceding  Process 

Assessment  Process 

Sub-process  10:  Progress  Against  Requirements 

Solution  Definition  Process 

Sub-process  18:  Physical  Solution  Representations 

Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 

System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 
Sub-process  31:  End  Product  Verification 

End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Inputs 

•  Deficiencies  and  discrepancies  (SP  1 0) 

•  Selected  physical  solution  representation  (SP  18) 

•  Requirement  statements  validation  revisions  (SP  25) 

•  Design  solution  deficiency  and  discrepancy  reports  (SP  30) 

•  End  Product  deficiency  and  discrepancy  reports  (SP  31) 

•  Operational  Test  /  Follow-On  Test  &  Evaluation  (OT/FOT&E)  Report  (SP  33) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  to  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 

following: 

a)  Fully  characterize  the  design  solution. 

b)  Ensure  that  the  design  solution  is  consistent  with  its  source  requirements  (selected  physical 
solution  representation  requirements,  associated  system  technical  requirements,  and  derived  technical 
requirements)  in  accordance  with  Sub-process  30.  Progress  against  requirements  will  be  re-entered 
when  discrepancies  and  deficiencies  are  identified  in  Sub-process  10  or  Sub-process  33.  This  task 
will  evaluate  for  appropriate  action  in  the  system  design  process. 

c)  Specify  requirements  (including  functional  and  performance  requirements,  physical  characteristics, 
and  test  requirements)  for  the  system,  system  end  products,  and  subsystems  of  each  end  product, 
as  applicable  to  the  engineering  life-cycle  phase,  in  accordance  with  Sub-process  25. 

d)  Record  the  design  solution  work  products,  including  the  specified  requirements,  in  the  established 
information  database  with  all  trade-off  analyses  results,  design  rationale,  assumptions,  and  key 
decisions  to  provide  traceability  of  requirements  up  and  down  the  system  structure. 

e)  Establish  projects  to  develop  enabling  products  and  to  procure  those  that  are  off-the-shelf  or  will 
be  reused,  that  will  satisfy  identified  requirements  for  the  associated  processes  (production,  test, 
deployment/installation,  training,  support  or  maintenance,  and  retirement  or  disposal)  related  to  the 
system’s  end  products. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Specified  Requirements  (SP  2,  3,  11,  20,  21,  25,  30,  31,  32)  (System,  subsystem,  and  interface 
specifications  that  describe  the  specified  requirements  (see  below))  in  the  form  of  an  Interface  Control 
Document  or  Detailed  Design  Specification 
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SSS  (System  /  Subsystem  Specification)  -  DI-IPSC-81431 
IRS  (External  Physical  Interfaces)  -  DI-IPSC-81434 
IRS  (Internal  Physical  Interfaces)  -  DI-IPSC-81434 

-  HWCI  (HW  Configuration  Item) 

SSDD  (System  Architecture  Design)  -  DI-IPSC-81432 
IDD  (HW  Interface  Design  Description)  -  DI-IPSC-81436 
Cl  Product  Descriptions 

SRS  (Software  Requirements  Specification)  -  DI-IPSC-81433 
CSCI  (CS  Configuration  Item) 

IRS  (Software  Interface  Requirements)  -  DI-IPSC-81434 
SDD  (Software  Design  Description)  -  DI-IPSC-81435 
DBDD  (DB  Design  Description)  -  DI-IPSC-81437 

-  IDD  (SW  Design  Description)  -  DI-IPSC-81436 

-  SPS  (SW  Product  Spec)  -  DI-IPSC-81441 

-  SVD  (User  SW  Version  Description)  -  DI-IPSC-81442 

•  Specified  Requirements  Products  (SP19) 

Parts  lists 

Procedural  manuals 

Data  and  other  applicable  design  descriptions 
Verified  design  solution 
Drawings/Schematics  (MIL-STD-100G) 

Supportability  Product  Specs/Descriptions 

•  Enabling  products  development  projects  (SP  32) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Sub-process  3 :  Supplier  Performance 
Assessment  Process 

Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Implementation  Process 

Sub-process  20:  Implementation 
Transition  to  Use  Process 

Sub-process  21:  Transition  to  Use 
Requirements  Validation  Process 

Sub-process  25:  Requirements  Statements  Validation 
System  Verification  Process 

Sub-process  30:  Design  Solution  Verification 
Sub-process  31:  End  Product  Verification 
Sub-process  32:  Enabling  Products  Readiness 

Agents 

Systems  Engineering,  R&M,  Human  Systems  Integration,  Safety,  Design,  Logistics,  Test,  Software 
Development 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 
Requirement  Allocation  Sheets  (RAS) 

Specification  Standards 

References 
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Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 
Systems  Engineering  Analysis  (Blanchard) 

Standard  Practice  for  Defense  Specifications  (MIL-STD-100G) 

Data  Item  Descriptions: 

System  /  Subsystem  Specification  (SSS)  (DI-IPSC-81431) 

Interface  Requirements  Specification  (IRS)  (DI-IPSC-81434) 

System  Architecture  Design  (SSDD)  (DI-IPSC-81432) 

Software  Requirements  Specification  (SRS)  (DI-IPSC-81433) 

Software  Design  Description  (SDD)  (DI-IPSC-81435) 

Database  Design  Description  (DBDD)  (DI-IPSC-81437) 

Interface  Design  Description  (IDD)  (DI-IPSC-81436) 

Software  Product  Specification  (SPS)  (DI-IPSC-81441) 

User  Software  Version  Description  (SVD)  (DI-IPSC-81442) 

Metrics  and  Measures 

•  Percent  Completion  of  Specified  Requirements  Products 

•  Percent  of  Specified  Requirements  Products  that  have  been  validated. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  a  fully  characterized  design  solution  that:  (1)  can  be  implemented 
through  further  development  of  subsystems,  off-the-shelf  procurement  or  reuse,  coding,  or  fabrication;  and  (2) 
provide  the  basis  for  the  assembly  and  integration  of  subsystem  products  into  end  products  required  for 
verification. 


NOTE  -  A  fully  characterized  design  solution  can  be  in  terms  of,  as  appropriate:  (1)  specifications  for  the 
system,  end  products,  subsystems,  and  applicable  interfaces;  (2)  interface  control  drawings  or  descriptions, 
detailed  drawings,  or  sketches;  and  (3)  parts  lists,  data  dictionaries,  or  other  planned  physical  configuration 
records. 
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4.4  Product  Realization 


The  Product  Realization  Processes  are  used  to:  (1)  convert  the  specified  requirements  and  other  design  solution 
characterizations  into  either  a  verified  end  product  or  a  set  of  end  products  in  accordance  with  the  agreement 
and  other  stakeholder  requirements;  (2)  deliver  these  to  designated  operating,  customer,  or  storage  sites;  (3) 
install  these  at  designated  operating  sites  or  into  designated  platforms;  and  (4)  provide  in-service  support,  as 
called  for  in  an  agreement. 

The  two  processes  related  to  Product  Realization  are  shown  in  Figure  4.4. 


Other  Stakeholder  Satisfaction 

Figure  4.4  -  Product  Realization  Process 

4.4.1  Implementation  Process 

One  sub-process  is  associated  with  the  Implementation  Process.  It  requires  transforming  the  characterized 
design  (preliminary  or  final)  into  an  integrated  end  product  that  conforms  to  its  specified  requirements. 


Sub-process  20-  Implementation 

The  developer  shall  implement  (build/assemble/code/test)  the  design  (preliminary  or  final)  in 
accordance  with  the  specified  requirements  to  obtain  a  verified  end  product. 


Preceding  Process 
Supply  Process 
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Sub-process  1 :  Product  Supply 

Planning  Process 

Sub-process  7:  Technical  Plans 

Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 

End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Inputs 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Manufacturing  Plans  (SP  7) 

•  Quality  Assurance  (QA)  Program  Plan  (SP  7) 

•  Specified  Requirements  (SP  19) 

•  Operational  Test  /  Follow-On  Test  &  Evaluation  (OT/FOT&E)  Report  (SP  33) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 

following. 

a)  Receive  from  suppliers,  reuse  from  off-the-shelf  supply,  or  receive  from  the  acquirer  (e.g., 

customer- furnished  items)  the  subsystem  products  that  make  up  the  system’s  end  products,  or,  as 
appropriate,  code  or  build  the  end  products  (software/hardware)  according  to  the  specified 
requirements  and  detailed  drawings  or  other  design  documentation.  Tools  for  this  task  includes  the 
following:  Parts  List,  Parts  Management  Plan,  Configuration  Item  Lists,  Make/Buy  Analysis, 
integrated  architecture  products  and  Government  Furnished  Equipment  (GFE)  Management.  This 
includes  ensuring  when  we  are  responsible  for  building  the  end  product;  and  ensuring  each  component 
or  piece  part  meets  its  specification. 


NOTE  -  Sub-process  3,  Supplier  Performance,  is  invoked  whenever  subsystem  products  are  acquired 
from  suppliers  or  lower-tier  developers  outside  the  enterprise,  as  well  as  when  the  supplier  is  an 
organizational  entity  within  the  developer’s  own  enterprise. 


b)  Validate  the  subsystem  products  received  or  reused  against  their  acquirer  requirements  (input 
requirements  to  the  subsystem  product  development)  using  the  End  Products  Validation  Process,  Sub¬ 
process  33,  unless  (1)  the  supplier  validated  the  products  prior  to  delivery  as  required  in  the 
agreement,  or  (2)  the  reused  products  have  already  been  validated.  Proof  of  validation  is  needed  for 
both  conditions.  Approval  of  Suppliers’  products  is  obtained  through  compliance  to  product 
specifications.  This  could  be  ascertained  at  suppliers’  facilities,  receiving  incoming  or  via  receipt 
inspection,  first  article  validation,  and/or  test/demonstration.  See  ISO  9001  Section  4.6.2  for  vendor 
management. 

c)  Assemble  the  validated  subsystem  products,  or  physically  integrate  such  products  into  the 
respective  test  article  or  end  product  to  be  verified.  This  should  be  accomplished  through  already 
approved  Manufacturing  and  Quality  Assurance  Program  Plans. 

d)  Verify  each  test  article  or  end  product  against  its  specified  requirements  (output  requirements  of 
the  system  end  product  development)  in  accordance  with  Sub-processes  30  and  31 .  Developmental 
Test  &  Evaluation  (DT&E)  (Sub-process  31)  accomplishes  such  a  task  for  end  products,  and  the  design 
solution  verification  (Sub-process  30)  does  this  for  test  articles  (brassboards). 
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e)  Ensure,  in  accordance  with  Sub-process  32„  that  the  enabling  products  for  each  associated  process 
will  be  ready  and  available  to  perform  their  intended  support  functions  required  by  the  system’s 
end  products.  An  area  often  missed  is  confirming  dedication  of  fleet  assets.  This  should  include 
CINC  notification  for  fleet  testing  as  well  as  GFE  and  other  assets  that  will  be  required. 


NOTE  -  The  relevant  end  products  for  enabling  products  are  verified  and  validated  as  necessary 
during  the  development  of  the  building  block  related  to  the  enabling  product  (see  Section  6).  All 
essential  systems  engineering  technical  reviews  (ITR,  ASR,  SRR,  TRA,  SFR,  PDR,  CDR,  TRR,  FRR, 
SVR/PRR,  PCR,  ISR,  etc.)  should  be  completed  to  ensure  that  enabling  processes  and  resources  are 
ready  and  available. _ 

A  major  Production  Readiness  Review  (PRR)  is  conducted  at  the  end  of  SDD  to  ensure  that  the 
program  is  ready  to  proceed  into  Low  Rate  Initial  Production  (LRIP).  This  review  will  validate  the 
production  facility,  equipment,  manufacturing  processes,  and  personnel;  and  help  ensure  that  the 
program  will  enter  low  rate  production  at  a  low  risk.  A  subsequent  PRR  is  usually  conducted  in  LRIP 
to  ensure  the  program  is  ready  to  transition  from  low  rate  to  full  rate  production  in  Production  & 
Deployment  and  Operations  &  Support  phases. 

f)  Validate  the  verified  end  products  against  their  acquirer  requirements  (input  requirements  to 
system  end  product  development)  prior  to  delivery,  if  required  by  the  agreement,  in  accordance  with 
Sub-process  33.  Operational  Test  &  Evaluation  (OT&E)  accomplishes  such  a  task  and  this 
information  is  incorporated  into  the  End  Product  or  Enabling  Product  Report. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Assembled  End  Product(s)  or  Enabling  Product(s)  (SP  20) 

•  Manufacturing  Process  &  Personnel  System  (SP  21) 

•  Verified  and  Validated  Integrated  End  Product  or  Enabling  Product  Report  (SP  21) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Transition  To  Use  Process 

Sub-process  21:  Transition  to  Use 

Agents 

Prime  Contractor,  Suppliers,  Program  Management,  Systems  Engineering,  Manufacturing,  Quality  Assurance 
(QA),  Logistics,  Testing,  Financial  Management,  Procurement,  Parts  Management,  End  User,  Defense 
Contractor  Management  Agency  (DCMA) 

Tools 

Design  tools;  Integrated  Enterprise  Data  Repository;  Manufacturing  Tooling;  Technical  Performance 
Measurement  (TPM)  Tracking  Tools/Schedules  (Earned  Value  Management  -  Schedule  Performance  Index 
(SPI)/Cost  Performance  Index  (CPI),  Availability  Metrics,  Reliability  Metrics,  and  Effectiveness  Metrics);  Test 
Equipment,  Test  Requirements,  Test  Analysis;  First  Article  Testing,  Manufacturing  Plan,  Work  Instructions, 
Statistical  Process  Control  (SPC);  Inspections,  In-Process  Inspection  Plan,  Production  Process  Flows, 
Integration  Control  Document,  Configuration  Systems  (Work  Breakdown  Structure);  and  Production  Readiness 
Review  (PRR),  Physical  Configuration  Review  (PCR).  Requirements  Management  &  System  Architecture 
Database  (ex.  CORE™,  DOORS,  SLATE™),  Parts  List,  Parts  Management  Plan,  Configuration  Item  Lists, 
Make/Buy  Analysis,  Government  Furnished  Equipment  (GFE)  Management,  and  integrated  architecture 
products. 

References 

Standard  across  all  systems  engineering  efforts: 
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•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

FAR  Parts  46  and  52.246 
DFAR  Part  46 
ISO  9001 

Defense  Manufacturing  Guide 
MIL-STD-1528A 
MIL-STD-1521B 
DOD-STD-2168 
DOD-STD-2 1 67A 

Systems  Engineering  Technical  Reviews  (SETRs) 

Manufacturing  Management  Program 

Metrics  and  Measures 

•  Adherence  to  Schedule  and  Progress  Versus  Plan 

•  Requirement  Execution  Time  and  Cost 

•  System  Definition  Detail 

•  Technical  Performance  Measurement  Resolution  (Availability,  Reliability,  Capability,  and  Effectiveness) 

•  Process  Control  Matrices 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  a  fully  integrated  end  product  that:  (1)  satisfies  its  specified 
requirements;  and  (2)  if  required  to  be  validated  prior  to  delivery,  conforms  to  its  related  acquirer  requirements. 

End  product  physical  integration  should  ensure  that:  (1)  internal  and  external  interfaces  for  the  composite  end 
product  (including  systems;  voice/data/information;  and  Doctrine,  Organization,  Training,  Material,  Leadership 
and  People,  and  Facilities  (DOTMLPF))  function  according  to  specified  requirements;  (2)  defined  states, 
modes,  dynamic  allocations  or  other  operational  switching  functions  perform  as  required;  and  (3)  and  designed 
overload  conditions,  reduced  operational  levels,  or  designed-in  degraded  mode  of  operations  are  included. 


4.4.2  Transition  to  Use  Process 

The  Transition  to  Use  Process  results  in  products  delivered  to  the  appropriate  destinations  in  the  required 
condition  for  use  by  the  acquirer  and  for  the  appropriate  training  of  installers,  operators,  or  maintainers  of  the 
products. 


Sub-process  21  -  Transition  to  Use 

The  developer  shall  transition  verified  products  to  the  acquirer  of  the  products  in  accordance 
with  the  agreement 


Preceding  Process 
Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Planning  Process 

Sub-process  6:  Schedule  and  Organization 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
Implementation  Process 


101 


Sub-process  2 1 


Sub-process  20:  Implementation 
System  Verification  Process 

Sub-process  32:  Enabling  Products  Readiness 

Inputs 

•  Verified  and  Validated  Integrated  End  Product  or  Enabling  Product  Report  (SP  20) 

•  Manufacturing  Process  &  Personnel  System  (SP  20) 

•  Integrated  Master  Schedule  (IMS)  (SP  6) 

•  Specified  Requirements  (for  packaging  and  handling)  (SP  19) 

•  Enabling  Products  Readiness  determination  (SP  32) 

•  ILS  Certification  (SP  2) 

•  Signed  DD  Form  250  (SP  2) 

Entry  Criteria 

Inputs  have  been  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Acquire  and  put  in  place  appropriate  enabling  products  to  carry  out  relevant  transition  to  use 
requirements.  Enabling  products  specifically  looked  for  are: 

•  Delivery  addresses 

•  Fleet  release  message 

•  Installation  procedures 

•  Training 

•  Operation  and  maintenance  manuals  (PHS&T) 

•  In-service  support  equipment 

b)  Prepare,  as  required  by  the  agreement,  end  products  for  shipping  and  storage. 

c)  Store  end  products  awaiting  shipping  and,  in  accordance  with  the  agreement,  ship  or  transport  to 
the  acquirer  at  the  intended  usage  sites. 

d)  Prepare,  as  required  by  the  agreement,  sites  where  end  products  will  be  stored,  installed,  used  or 
maintained,  or  serviced. 

e)  Install  end  products,  as  required  by  the  agreement,  at  the  appropriate  sites. 

f)  Perform  commissioning,  as  required  by  the  agreement,  to  bring  delivered  or  installed  end  products 
to  operational  readiness  with  appropriate  acceptance  and  certification  tests  completed  in 

accordance  with  Sub-process  33. 

g)  Provide,  if  required  by  the  agreement,  a  parallel  operation  (ghosting)  of  the  new  and  the  legacy  end 
products  so  that  service  is  continuous  during  the  transition  period. 

h)  Provide,  in  accordance  with  the  agreement,  training  for  users,  maintenance,  and  other  personnel. 

i)  Provide,  in  accordance  with  the  agreement,  in-service  support. 

j)  Deliver  all  planned  support  elements. 

Outputs  (“EXT”  indicates  it  is  external,  unspecified,  and  not  for  a  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Operational  system  products  (EXT) 
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Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 

Agents 

Logistics 

Fleet  Support  Team  (FST) 

In-service  Support 
PM 

Tools 

Not  Applicable 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 


Metrics  and  Measures 
Percent  damaged  products 
On-time  delivery 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  fulfill  the  delivery  requirements  of  the  agreement. 


NOTE  -  Transition  to  Use  tasks  will  be  dependant  on  whether  the  end  product  is  being  delivered  for 
intended  marketplace  use  or  sale,  or  if  the  end  product  is  delivered  to  another  developer  for  integration  into 
a  set  of  other  end  products  to  make  up  an  end  product  higher  in  the  system  structure. 


4.5  Technical  Evaluation 

The  Technical  Evaluation  Processes  are  intended  to  be  invoked  by  one  of  the  other  processes  for  engineering  a 
system.  Four  processes  are  involved:  Systems  Analysis,  Requirements  Validation,  System  Verification,  and 
End  Products  Validation.  The  relationship  between  these  processes  is  shown  in  Figure  4.5. 
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Analysis  Requests,  Requirements,  Implemented  Products 


Systems 

Analysis 

Process 


Products 

Characteristics 


Verification  Results 


Validation  Results 


Requirements 

Validation 

Process 


Val 


Requirement 
Conflicts  &  Issues 


Systems 

Verification 

Process 


End  Products 

Validation 

Process 


Analytical  Models  &  Assessments,  Validated  Requirements, 
Verified  System  Products,  Validated  End  Products 

Figure  4.5  -  Technical  Evaluation  Process 


4.5.1  System  Analysis  Process 

The  Systems  Analysis  Process  is  used  to:  (1)  provide  a  rigorous  basis  for  technical  decision  making,  resolution 
of  requirement  conflicts,  and  assessment  of  alternative  physical  solutions;  (2)  determine  progress  in  satisfying 
system  technical  and  derived  technical  requirements;  (3)  support  risk  management;  and  (4)  ensure  that  decisions 
are  made  only  after  evaluating  the  cost,  schedule,  performance,  and  risk  effects  on  the  engineering  or 
reengineering  of  the  system. 

The  three  sub-processes  associated  with  the  Systems  Analysis  Process,  when  invoked  by  other  processes  in  this 
Guide,  are  shown  in  Figure  4.5.1a. 


System 

Analysis 

Process 

Requirements 


Sub-process  22  -  Effectiveness  Analysis 

Sub-process  23  -  Trade-off  Analysis 

Sub-process  24  -  Risk  Analysis 


Figure  4.5.1a  -  System  Analysis  Process/Sub-processes 


Completion  of  Systems  Analysis  sub-processes  should  ensure,  as  appropriate,  that: 


a)  the  effectiveness  of  each  design  solution  is  appropriately  evaluated; 
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b)  the  effect  on  any  interfacing  products  or  platforms  is  evaluated  for  each  alternative  solution  in  time  to 
support  selection  among  these  alternatives  (this  includes  interoperability  and  integration  effects  of 
electronic  interference  and  communication,  as  well  as  functional,  human,  and  physical  interfaces; 

c)  cost  (e.g.,  unit  production  cost,  developmental  cost,  and/or  life  cycle  cost)  is  appropriately  treated  as  an 
assigned  requirement  or  as  an  independent  variable  when  conducting  trade-offs  with  technical 
requirements; 

d)  cost  or  price,  schedule,  performance,  and  risk  effects  of  each  functional,  performance,  and  design 
alternative  are  defined,  calculated,  and  reported; 

e)  estimated  total  ownership  costs  including  hidden  cost  effects  (for  example,  from  manufacturing 
processes  variability;  excessive  precision  of  manufacturing  or  test  processes;  special  materials, 
finishes,  and  painting  of  products;  and  product  complexity),  the  cost  of  operation,  and  all  associated 
processes  are  known; 

f)  primary  functional  characteristics  of  solutions  (for  example,  producibility,  testability,  deployability, 
operability,  supportability,  trainability,  and  disposability)  are  directly  traceable  to  the  functional  and 
performance  requirements  they  were  designed  to  fulfill; 

g)  applicable  product  dependability  factors  such  as  availability,  maintainability,  reliability,  safety,  and 
security  are  not  degraded; 

h)  projected  environmental  impacts  are  known; 

i)  design  assumptions  are  valid  and  reasonable; 

j)  technology  limits  are  recognized  and  understood;  and 

k)  requirements  can  be  validated  and  specifications  verified  in  a  cost-effective  manner. 

The  developer  should  identify,  acquire/develop,  and  implement  models,  including  prototypes  and  simulations  as 
applicable,  to  accomplish  effectiveness  analyses,  do  trade-off  analyses,  and  complete  risk  analyses  invoked  by 
processes  in  this  Guide. 

The  effectiveness  analysis  requirement  is  an  integral  part  of  a  trade-off  analysis  (Sub-process  23).  It  can  also 
be  done  as  needed  to  analyze  the  effectiveness  of  the  preferred  solution  selected  during  the  activities  of  Sub¬ 
process  18,  Physical  Solution  Representations.  Also,  effectiveness  analyses  should  be  used  to  support  risk 
impact  analyses  and  requirements  definition  in  general. 

Figure  4.5.1b  illustrates  the  interrelationship  of  Effectiveness  Analysis  with  trade-off  Analysis  (Sub-process 
23)  and  Risk  Analysis  (Sub-process  24).  One  can  do  a  risk  analysis  without  also  doing  an  effectiveness 
analysis  or  trade-off  analysis  (e.g.,  for  doing  risk  management  -  Sub-process  12).  However,  an  effectiveness 
analysis  can  also  be  done  to  support  a  risk  impact  assessment.  One  can  do  an  effectiveness  analysis  without 
doing  a  risk  analysis  (e.g.,  for  the  physical  solution  definition  -  Sub-process  18).  However,  one  does  not  do  a 
trade-off  analysis  without  also  doing  both  a  risk  analysis  and  an  effectiveness  analysis  (as  illustrated  by  the  line 
O-A  in  Figure  4.5.1b).  Thus  point  A  can  be  anywhere  in  the  quadrant  space  except  along  the  trade-off  analysis 
axis.  The  degree  of  risk  analysis  and  effectiveness  analysis  can  vary  depending  on  the  case  invoking  the  trade¬ 
off  analysis. 
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I 

TRADE-OFF  A 


Trade-off  analyses  provide  a  recommended  course  of  action  with  impact  to  decision  makers  for  each  set  of 
alternatives  examined.  The  impact  assessment  is  given  in  terms  of  cost,  schedule,  performance  and 
risk/opportunity. 

Opportunity  analysis  should  be  added  to  risk  analysis.  Opportunity  analysis  is  important  to  include  in  a  trade-off 
analysis  because  an  opportunity  offered  by  an  alternative  may  be  a  driving  force  in  making  a  recommendation. 
Opportunities  exist  in  terms  of  the  capacity  to  evolve.  That  is  looking  at  the  capacity  to  add  improvements  at  a 
later  date  in  order  to  improve  competitiveness  or  to  overcome  an  evolving  threat  to  do  technology  refreshment 
that  will  make  the  product  more  efficient  or  cost  effective,  or  to  take  advantage  of  a  technology  insertion  that 
will  improve  performance.  Opportunities  also  exist  to  be  able  to  evolve  one  configuration  into  another  without 
a  new  development  effort.  (One  of  the  basics  of  evolutionary  development.) 

Effectiveness  can  be  looked  at  as  the  measure  of  extent  to  which  a  system,  or  portion  of  a  system,  may  be 
expected  to  achieve  a  set  of  specific  objectives  based  on  the  alternative  attribute  or  solution  being  analyzed. 
These  objectives  may  be  based  on  a  mission  profile,  concept  of  operation,  or  overall  functionality  requirements 
of  the  system,  or  portion  of  the  system,  being  analyzed.  The  analysis  provides  a  quantitative  determination  of 
how  well  the  resulting  end  product  would  meet  four  metrics  -  capability,  dependability,  suitability,  and  cost 
effectiveness.  (See  metrics  at  the  end  of  Sub-process  22.) 

The  economic  consequences  of  an  alternative  selection  are  an  important  consideration  in  decision-making. 
Therefore,  in  doing  a  trade-off  analysis  during  development,  sufficient  cost  trade-off  data  should  be  available  to 
the  decision  maker  so  that  alternatives  can  be  compared  in  terms  of  capability,  dependability,  suitability  criteria, 
as  well  as  risk  and  opportunity. 

Figure  4.5.1c  provides  the  input  -  output  relationships  of  Sub-processes  22,  23  and  24  as  defined  in  this  guide. 
Additionally,  in  dashed  lines  are  the  recommended  relationship  of  Sub-processes  22  and  24  for  situations  when 
risk  analysis  is  done  without  Sub-process  23  implementation. 
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Invokes 


Figure  4.5.1c  -  Interrelationship  of  Systems  Analysis  Sub-processes 


Figure  4.5. Id  provides  the  sub-processes  that  invoke  each  Systems  Analysis  Process  and  it  provides  the  sub¬ 
processes  that  may  receive  the  outputs  of  each  Systems  Analysis  Sub-process  as  defined  in  this  guide.  An 
added  trigger  and  output  destination  for  Sub-processes  22  and  24  are  shown  in  parenthesis. 


Invoked  by  Sub-process 
12,  18,  23 

-0- 

Sub-process  24 
Risk  & 
Opportunity 
Ana^sis 

Outputs  to 
12,  18,  (22),  23 


Invoked  by  Sub-process 
,18 


Sub-process  23 

T  radeoff 
Analysis 

— ri — 

Outputs  to 
12,  16, 17,  18 


Invoked  by  Sub-process 
14,  15,  16,  17,  18,  23 

-q 

Sub-process  22 
Effectiveness 
Analysis 

5 

Outputs  to 

12,  14,  15,  16,  17,  18,  23 


Note:  Although  tradeoff  analysis  is  not  explicitly  call  out  while  fully  characterizing  the  design 
solution  in  Sub-process  19,  such  may  need  to  be  done. 

Figure  4.5. Id  -  Sub-processes  Invoking  or  Receiving  Outputs  from  Systems  Analysis  Sub-processes 


Sub-process  22  -  Effectiveness  Analysis 

The  developer  shall  perform  effectiveness  analyses  to  provide  a  quantitative  basis  for 
decision-making. 


Effectiveness  analyses  are  done  to:  (1)  measure  the  extent  each  alternative  physical  solution  considered  during 
design  may  be  expected  to  achieve  system  requirements;  (2)  assist  in  choosing  the  preferred  physical  solution 
for  the  end  product  being  developed;  and  (3)  aid  in  determining  recommended  courses  of  action  and  associated 
impacts  for  trade-off  analyses.  Effectiveness  analyses  are  also  used  during:  (1)  System  Technical  Requirements 
definition  to  support  performance  analyses  to  determine  a  “knee  in  the  curve”  or  some  other  identifiable 
characteristic  that  provides  an  optimal  set  of  requirements;  (2)  Progress  Against  Requirements  Assessments  to 
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determine  how  well  the  design  solution  is  maturing  toward  meeting  agreement  requirements;  and  (3)  Technical 
Reviews  for  providing  the  review  decision  makers  with  the  maturity  of  the  design  solution. 

Preceding  Process 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Sub-process  15:  Other  Stakeholder  Requirements 
Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 

Inputs 

•  Technical  Performance  Measures  (TPM)  (SP  5) 

•  CAIV  decision  criteria  (SP  5) 

•  System  Engineering  Plan  (SEP)  or  Software  Development  Plan  (SDP)  (SP  7) 

•  Effectiveness  Analysis  Request  (SP  14,  15,  16,  17,  18,  23) 

An  Effectiveness  Analysis  request  can  come  from:  (1)  Sub-processes  14  or  15  to  request  effectiveness  analyses 
to  aid  in  developing  the  understanding  of  customer  requirements  (i.e.,  mission  analyses,  measures  of 
effectiveness;  and  operational  concept  analyses);  (2)  Sub-process  16  to  aid  in  developing  the  best  set  of 
technical  performance  requirements;  (3)  Sub-process  17  to  aid  in  forming  logical  solution  representations  and 
derived  technical  requirements;  (4)  Sub-process  18  to  aid  in  defining  alternative  physical  solution 
representations  or  alternative  attributes  for  a  single  physical  solution  representation  or  in  selecting  the  preferred 
solution;  and  (5)  Sub-process  23  to  aid  in  doing  a  trade-off  analysis. 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Request  for  effectiveness  analysis  is  clear,  concise,  and  valid;  the  input  descriptive  data  is  complete  and 
consistent  for  the  effectiveness  analysis  to  be  done;  appropriate  resources  for  doing  the  analysis  are  available; 
appropriate  models  and/or  simulations  are  defined  and  available;  and  completion  time  constraints  are  defined 
and  acceptable.  In  addition,  the  roles,  responsibilities  and  authorities  needed  to  do  effectiveness  analyses 
should  be  identified  and  defined,  as  well  as  assignment  of  the  roles,  responsibilities,  and  authorities  to  the 
appropriate  team  or  individual. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include  the 
following: 

a)  Plan  effectiveness  analyses  to  include  purpose,  objectives,  execution  and  data  collection,  schedule 
of  tasks,  resource  need  and  availability,  and  expected  outcomes. 

The  plans  for  doing  effectiveness  analyses  should  be  done  in  conjunction  with  planning  for  systems 
analysis  and  include  definition  of  any  special  techniques,  procedures,  tools  needed,  and  simulations 
and  modeling. 

Effectiveness  models  should  be  created  for  specific  characteristics  of  system  functionality.  These 
characteristics  include,  but  are  not  limited  to:  operations  (such  as  measures  of  effectiveness), 
supportability,  reliability,  maintainability,  production,  training,  disposal,  test/validation/verification, 
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deployment/installation,  environmental,  and  total  ownership  cost  (including  design  to  cost  or  cost  as  an 
independent  variable).  Effectiveness  models  should  allow  parameters  to  be  varied  so  that  relative, 
individual  effect  on  total  system  performance  and  life  cycle  cost  can  be  determined.  All  effectiveness 
models  must  be  validated  to  ensure  valid  analysis  and  simulation  results. 

b)  Analyze  each  alternative  for  system  and  cost  effectiveness  based  on  factors  such  as  accuracy, 

availability,  capacity,  maintainability,  reliability,  responsiveness,  operability,  safety,  security,  spares, 
requirements,  survivability,  transportability,  and  vulnerability.  Navy  Systems  Commands  have  areas 
specifically  assigned  to  engineering  specialty  such  as  ILS,  producibility,  and  deployability.  Often 
there  are  full  plans  specifically  covering  these  areas  when  they  are  vital  to  the  program  development 
(see  Sub-process  7  and  Appendix  D).  These  plans  provide  further  context  for  planning  effectiveness 
analyses. 

Cost  may  be  treated  like  a  performance  objective  (design  to  cost)  or  as  an  independent  variable 
(CAIV).  System  and  cost  effectiveness  analyses  should  include  the  following,  as  applicable: 

1)  Production  engineering  analysis  and  assessment  to  determine  what  it  will  take  to  manufacture  or 
produce,  including  assembly  and  integration,  the  resulting  end  product.  This  includes: 
producibility-related  design  factors;  alternative  manufacturing  and  production  approaches;  impacts 
of  long-lead-time  items;  and  material,  capacity,  tools,  equipment,  and  people  limitations. 

2)  Test  and  evaluation  analysis  and  assessment  to  determine  what  it  will  take  to  do  necessary  tests 
and  evaluations  on  the  resulting  end  products.  This  includes:  analyzing  the  various  kinds  of 
validations,  verifications,  demonstrations,  qualification,  acceptance  and  other  testing  that  may  be 
needed;  testability-related  design  factors;  and  test  and  evaluation  requirements  such  as  testing 
sites,  facilities,  site/facility  capacities  and  limitations,  people,  and  life-cycle  testing  consistency. 

3)  Deployment  and  installation  analysis  and  assessment  to  determine  the  requirements  and 
constraints  associated  with  deploying  and/or  installing  the  resulting  end  product.  This  includes: 
factors  for  site/host  selection,  activation/installation,  on-site  assembly,  and  site-unique  hazards; 
compatibility  with  existing  infrastructures;  environmental  impact  considerations;  early 
deployment  of  training  items  and  personnel;  initial  provisioning  and  spares;  packaging,  handling, 
storage,  and  transportation  requirements  and  constraints;  and  site  transition  requirements. 

4)  Operation  analysis  and  assessment  to  determine  what  it  will  take  to  satisfy  operational 
requirements  for  the  resulting  end  product.  This  includes:  operation  and  support  facility  and 
equipment  requirements;  interoperability  of  interacting  systems  required  to  execute  operational 
functions  in  the  intended  use  environments;  required  joint  and  combined  operations  including 
other  services,  contractors  and  international  partners;  and  planned  and  potential  future  operation 
uses. 

5)  Supportability  analysis  and  assessment  to  determine  what  it  will  take  to  support  end  products  over 
the  life  cycle.  This  includes:  supportability-related  design  factors;  all  planned  levels  of 
maintenance;  and  support  resources  required  such  as  people,  parts,  facilities,  and  materials. 

6)  Training  analysis  and  assessment  to  determine  what  it  will  take  to  train  users  of  the  resulting  end 
product.  This  includes:  development  of  qualified  personnel  with  appropriate  skills,  proficiencies 
and  capabilities;  initial  and  follow-on  training  requirements;  and  training  resources  required  such 
as  people,  facilities,  training  materials,  and  how  often  re-training  will  be  required  (perishability  of 
previous  training). 

Determine  the  sensitivity  to  constraints  and  uncertainties  in  input  data  and  assumptions.  When  another 
system  has  comparable  characteristics,  it  can  be  used  as  a  baseline  to  support  the  determination, 
completeness,  and  achievability  of  effectiveness  analysis  requirements. 
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c)  Analyze  each  alternative  for  total  ownership  cost  to  the  enterprise  and  to  the  acquirer.  The 

following  costs  are  typically  included  in  a  total  ownership  cost  analysis:  development,  production,  test, 
deployment/installation,  training,  operations,  support/maintenance,  and  retirement/disposal. 

Of  interest  is  determining  the  economic  consequences  of  each  alternative  in  terms  of  costs  to  the 
enterprise  and  to  the  acquirer  for  each  alternative  physical  solution  representation,  alternative  trade-off 
analysis  option,  or  proposed  change.  As  a  result  of  this  analysis,  design-to-cost  targets  (if  applicable), 
current  estimate  of  system  total  life-cycle  cost,  and  known  uncertainties  in  these  costs  should  be 
established. 

d)  Analyze  the  environmental  impact  of  each  alternative,  including  applicable  environmental  statutes 
and  hazardous  material  lists,  from  an  enterprise-based  life  cycle  perspective  (see  Appendix  B). 

The  system  and  its  end  products  must  operate  within  prescribed  environmental  definitions.  The 
system/end  products  and  the  environment  will  interact  in  certain  ways,  and  the  goal  is  to  minimize  the 
adverse  impact  of  the  system/end  products  on  its  environment  and  the  environment  on  the  system/end 
products.  Environmental  impacts  should  include  the  natural  environment  (air,  land,  and  water), 
organizational  environment  (enterprise  and  geo-political),  and  social  environment  (people,  animal, 
plant,  cultures,  and  religions). 

It  is  important  to  understand  the  interfaces  between  the  system/end  products  and  the  environment  in 
terms  of  all  materials  and  energies  exchanged  across  the  interface.  Each  interface  is  studied  for  ways 
of  reducing  environmental  impact. 

Likewise,  environmental  laws  and  regulations  must  be  studied  for  compliance.  The  developer  must 
adhere  to  all  applicable  statutes  and  agreements  to  designated  hazardous  material  lists.  Use  of 
materials  that  present  a  known  hazard  will  be  avoided  to  the  extent  possible.  Legal  implications  to  the 
government  should  be  identified  and  defined. 

An  environmental  impact  analysis  should  include,  as  applicable: 

1)  Environmental  analysis  and  assessment  to  determine  the  impact  on  and  by  each  end  product  and 
enabling  product  alternative  on  factors  such  as  noise  pollution,  quantities  and  types  of  hazardous 
materials  used,  hazardous  waste  disposal,  and  other  defined  environmental  requirements 
applicable.  This  includes,  from  an  enterprise-based  life  cycle  perspective:  the  applicable  federal, 
state,  municipal,  and  international  environmental  statutes  and  applicable  hazardous  material  lists 
affecting  the  project;  endurance  of  compliance  by  each  physical  solution  end  product;  and  the 
effect  on  and  by  each  end  product  and  enabling  product  on  the  infrastructure,  land  and  ocean, 
atmosphere,  water  sources,  and  animal,  plant  and  human  life,  as  applicable. 

2)  Disposal  analysis  and  assessment  to  determine  what  it  takes  to  dispose  of  end  products  and  by¬ 
products.  This  includes:  disposability-related  design  factors;  identifying  environmental  factors  for 
process  wastes  and  outputs  as  well  as  used  end  products  and  their  subsystems;  consideration  of 
various  disposal  methods  such  as  storage,  dismantling,  demilitarization,  reusing,  recycling,  and 
destruction;  and  people,  costs,  sites,  responsible  agencies,  handling  and  shipping,  supporting 
items,  and  applicable  federal,  state,  local,  and  host  nation  regulations. 

e)  Analyze  each  alternative  for  each  required  operational  profile  to  provide  an  analytical 
confirmation  that  the  alternative  satisfies  appropriate  requirements.  This  task  uses  the  outputs  of 
tasks  b)  through  d)  above  as  inputs  to  analyze  each  alternative. 

For  analysis  of  alternative  physical  solution  representations  or  of  the  preferred  physical  solution, 
satisfaction  of  the  set  of  derived  technical  requirements  should  be  confirmed. 
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For  analysis  of  alternative  attributes  (for  requirement  conflict  resolution)  or  for  evaluating  logical 
solution  representations,  the  impact  on  the  ability  to  satisfy  the  defined  system  technical  requirements 
within  acceptable  costs  and  risks  should  be  considered. 

f)  Record  effective  analysis  outcomes  in  the  established  enterprise  data  repository,  including 
assumptions,  details  of  the  analysis,  findings,  lessons  learned,  models  used,  rationale  for  decisions 
made,  and  other  pertinent  information  that  affects  the  interpretation  of  the  effectiveness  analysis 
results. 

The  results  of  the  effectiveness  analysis  should  be  provided  to  the  requesting  source  and  recorded  in 
the  enterprise  data  repository  (Sub-process  12).  It  is  important  for  follow-on  analyses  that  models, 
data  files,  and  their  documentation  be  maintained,  updated  and  modified  as  required.  Each  version  of  a 
model  or  data  file  that  impacts  requirements,  design,  or  decisions  should  be  entered  into  the  enterprise 
data  repository. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Effectiveness  Analysis  Plan  (SP  22) 

•  Effectiveness  Models  (SP  14,  15,  16,  17,  18,  23) 

•  Production  Engineering  Assessment  (SP  22) 

•  Test  &  Evaluation  Assessment  (SP  22) 

•  Deployment  &  Installation  Assessment  (SP  22) 

•  Operations  Assessment  (SP  22) 

•  Support  Assessment  (SP  22) 

•  Training  Assessment  (SP  22) 

•  Total  Ownership  Cost  Assessment  (SP  22) 

•  Environmental  Assessment  (SP  22) 

•  Disposal  Assessment  (SP  22) 

•  Effectiveness  Analysis  Reports  (SP  14,  15,  16,  17,  18,  23,  24) 

Effectiveness  Analysis  Reports  are  provided  to  the  requestor  of  the  effectiveness  analysis  and  captured  in  the 
enterprise  data  repository.  Each  report  will  document  the  results  of  the  effectiveness  analysis  in  accordance 
with  the  agreement  and  effectiveness  analysis  plan  to  include:  outcomes  from  each  analysis  and  assessment 
made  and  who  approved  the  results;  input  data  used  and  who  approved  the  data;  models  used;  and  related  data 
files,  assumptions,  and  lessons  learned.  Some  examples  of  types  of  reports/analyses  may  include  Mission  Area 
Analysis  (MAA),  Measures  of  Effectiveness  (MOE),  Mission  analysis,  Analysis  of  Alternatives,  System 
concept  analysis,  etc. 

For  effectiveness  analyses  that  support  Sub-processes  14  or  15  -  Acquirer  requirement  and  other  stakeholder 
requirements  are  analyzed  to  determine  warfighter  deficiencies  and  to  analyze  technology  opportunities  for 
increased  systems  effectiveness  and/or  cost  reductions. 

For  effectiveness  analyses  that  support  Sub-process  16  -  System  Technical  Requirements,  outcome  data 
includes: 

•  the  effectiveness  of  various  mixes  of  requirements  without  regard  to  the  means  of  implementation  (except 
for  legacy  systems  for  which  changes  of  performance  are  being  considered),  and 

•  effectiveness  to  help  come  up  with  a  “knee  in  the  curve”  or  some  other  identifiable  characteristic  that 
provides  an  optimal  set  of  requirements. 

For  effectiveness  analyses  that  support  Sub-process  17  -  Logical  Solution  Representation,  the  outcome  data  are 
very  similar  to  those  for  Sub-process  16  in  that  effectiveness  of  various  logical  representations  are  considered 
without  regard  to  the  means  of  implementation  (except  for  legacy  systems). 

For  effectiveness  evaluations  to  support  trade-off  analyses  of  alternative  physical  solution  representations  or  an 
evaluation  of  the  preferred  physical  solution  (Sub-process  18),  the  outcome  data  provides  a  quantitative 
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assessment  of  the  value  of  a  point  design  solution.  The  objective  of  these  evaluations  is  to  measure  how  well 
the  point  design  meets  its  set  of  derived  requirements.  For  systems  effectiveness  assessments  that  support  Sub¬ 
process  18,  outcome  data  includes,  as  applicable: 

•  overall  system  or  system  product  effectiveness  for  each  operational  profile  with  respect  to  satisfying 
acquirer  requirements  within  acceptable  risks 

•  impact  on  enabling  product  requirements  with  respect  to  each  associated  process  (development  and 
integration,  production/manufacturing,  test,  deployment,  training,  operations,  support,  and  disposal) 

•  system  cost  effectiveness  with  respect  to  attributes  such  as:  capability  (accuracy),  dependability 
(availability,  reliability,  operability,  survivability,  and  vulnerability),  and  suitability  (capacity, 
maintainability,  responsiveness,  safety,  security,  spare  requirements,  and  transportability) 

•  total  ownership  costs  to  the  enterprise,  acquirer,  and/or  user,  including  the  known  uncertainties  (risks)  in 
these  costs 

•  compliance  impacts  of  applicable  federal,  state,  municipal,  and  international  environmental  statutes  and 
applicable  hazardous  material  lists,  as  well  as  legal  liabilities 

•  environmental  impacts  on  the  land  and  ocean,  atmosphere,  water  sources,  and  animal,  plant  and  human  life. 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

For  analysis  of  alternative  physical  solution  representations  or  of  the  preferred  physical  solution,  satisfaction  of 
the  set  of  derived  technical  requirements  should  be  confirmed  within  acceptable  levels  of  risk  and  within 
acceptable  costs. 

For  analysis  of  alternate  attributes  for  requirement  definition  conflicts  or  logical  solution  representations, 
satisfaction  of  the  defined  set  of  technical  requirements  for  the  system  should  be  confirmed  within  acceptable 
levels  of  risk  and  within  acceptable  costs. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Sub-process  15:  Other  Stakeholder  Requirements 
Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  23:  Trade-off  Analysis 
Sub-process  24:  Risk  Analysis 

Agents 

Legal 

Systems  Engineering 

(The  Systems  Engineering  Manager  is  the  primary  agent  for  approval  of  inputs  to  effectiveness  analysis  and  for 
approving  effectiveness  analysis  outputs.) 

Tools 

Effectiveness  models  and  integrated  architecture  products  should  be  used  when  they  can  contribute  to  the 
decision  process.  Effective  models  allow  parameters  to  be  varied  so  that  their  relative,  individual  effect  on  total 
system  performance  or  end  product  performance  and  life  cycle  cost  can  be  determined.  Specific  models  will 
depend  on  the  system  or  end  product  being  analyzed,  its  size,  its  location  in  the  total  system  architecture,  and 
the  phase  of  development  within  the  DoD  Acquisition  Process.  Early  effectiveness  modeling  during  feasible 
concept  trade-off  studies  may  take  a  functional  view,  while  later  modeling  during  physical  design  trade-off 
analyses  may  shift  to  a  product  view. 
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Integrated  architectures  provide  a  logical,  structured  approach  for  defining  how  forces  operate  by  defining  the 
missions,  functions  and  tasks  required;  associated  systems/sub-system  functionality,  and  logistic  support.  They 
describe  the  Doctrine,  Organization,  Training,  Material,  Leadership  and  Education,  People,  and  Facilities 
(DOTMLPF)  and  their  relationship  in  term  of  information  flow  and  the  technical  standard  required  supporting 
the  missions.  Architecture  uses  can  include,  but  are  not  limited  to,  effectiveness  analysis;  developing 
requirement  and  derived  requirement  documents;  conduct  interoperability  reviews;  system  development  and 
integration;  and  resource  management. 

The  best  tool  to  use  for  effectiveness  analysis  is  a  simulation  model.  For  legacy  products,  a  simulation  model 
often  exists.  Well-constructed  simulation  models  are  useful  for  parametric  type  analysis  to  determine  the  effect 
of  varying  the  attributes  being  used  in  a  trade-off  or  for  analyzing  the  physical  solution  (alternatives  or 
preferred)  based  on  applicable  design  parameters. 

For  larger  legacy  systems,  virtual  reality  models  are  more  useful  for  form  and  fit  type  trade-offs  and,  depending 
on  related  algorithms,  for  determining  functional  performance  and  cost. 

For  new  systems  or  products,  mathematical  models  may  need  to  be  created  before  a  simulation  model  is 
applicable. 

Caution  is  needed  when  using  effectiveness  measures  and  their  models.  It  must  be  recognized  when  the 
outcome  of  system  effectiveness  is  uncertain.  Obtaining  trustworthy  relationships  among  the  system 
performance  and  system  effectiveness  is  often  difficult.  Models  often  only  treat  one  or  two  of  the  parameters  at 
a  time;  supporting  models  may  not  have  been  properly  integrated;  data  are  often  incomplete  or  unreliable,  and 
assumptions  may  not  be  valid.  Thus,  results  from  trade-off  analyses  using  such  models  and  measures  may  only 
express  relative  effectiveness  of  alternatives  within  the  context  of  the  trade-off  analysis. 

A  valid  effectiveness  model  should  provide  trustworthy  relationships  between  the  underlying  performance  and 
technical  attributes  involved  and  the  system  effectiveness  measure  of  interest.  The  effectiveness  measure  and 
its  measurement  model  must  be  tailored  to  the  maturity  of  the  system  design.  As  the  system  design  and 
operational  concept  mature,  effectiveness  estimates  should  mature  as  well. 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

MIL-STD-499B,  Systems  Engineering ,  October  1993  (not  formally  approved  by  OSD),  section  4. 3.4.2  and  5.5. 

Engineering  Complex  Systems  with  Models  and  Objects ,  David  W.  Oliver,  et  al,  1997,  Chapter  6  includes  a 
discussion  of  effectiveness  measures  and  models  related  thereto. 

Systems  Engineering  Guidebook ,  James  N.  Martin,  1996,  Section  7.4  includes  discussions  and  models  for 
performing  system  and  cost  effectiveness  analyses. 

Systems  Engineering  Management ,  James  Lacy,  1992,  Part  II  includes  specialty  engineering  considerations  for 
effectiveness  analyses. 

System  Engineering  Planning  and  Enterprise  Identity ,  Jeffery  O.  Grady,  1994,  Part  II,  Section  6  includes 
discussions  on  specialty  integration  considerations  that  help  in  doing  effectiveness  analyses. 

Models  and  Simulations:  DAU  Program  Manager's  Tool  Kit  includes  a  discussion  of  the  classification  of 
models  and  simulations. 

Virtual  Prototyping  -  Concent  to  Production ,  DSMC,  Report  of  the  1992-1993  Military  Research  Fellows, 
Chapter  3  includes  a  discussion  of  the  spectrum  of  synthetic  environments  with  listings  and  descriptions  of 
models. 

Metrics  and  Measures 
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Technical  Performance  Measurement  (TPM)  -  used  to  track  progress  toward  achieving  a  critical  parameter 
related  to  the  system.  The  critical  parameter  is  usually  a  measure  of  effectiveness  important  to  the  customer  in 
that  its  failure  to  be  achieved  will  cause  non-acceptance  of  the  system.  To  classify  as  a  TPM  measure,  the 
performance  parameter  must  be  a  significant  qualifier  or  determinant  of  the  total  system,  a  direct  measure  of 
value  that  can  be  derived  from  results  of  analyses  or  tests,  and  a  time-based  value  and  tolerance  band  that  can  be 
predicted  and  profiled  for  each  parameter  and  substantiated  during  development  and  test.  The  profile  is 
compared  with  a  threshold  value  which  if  not  attained  at  the  end  of  development,  or  if  fallen  below  or  above, 
whichever  case  is  unacceptable  for  the  selected  parameter,  a  Defense  Acquisition  Board  review  of  the  program 
is  mandated. 

Measures  of  Effectiveness  (MOEs)  are  the  critical  requirements  by  which  the  acquirer  will  determine  system 
acceptance.  Therefore,  these  measures  incorporate  the  following  essential  system  cost  and  effectiveness 
metrics: 

Capability  -  a  measure  of  an  end  product’s  (e.g.,  airplane,  or  missile)  ability  to  perform  the  tasks  for  which 
it  is  intended  in  the  environments  it  is  intended  to  operate  and  with  the  operator  or  user  level  of  skill 
intended,  given  that  the  end  product  is  dependable  and  suitable.  The  question  answered  is:  Will  it  get  the 
task  done?  The  end  product  must  be  able  to  complete  its  task  in  a  full  readiness  status  or  in  a  degraded 
status.  Capability  is  directly  related  to  the  operational  tasks  the  end  product  is  required  to  do  (e.g.,  destroy 
target,  communicate,  move  supplies  to  a  designated  destination,  or  obtain  required  information). 

Dependability  -  a  measure  of  the  degree  to  which  an  end  product  is  operable  and  available  to  perform  its 
required  function  at  any  given  (random)  time,  given  it  is  suitable  for  its  intended  use.  The  question 
answered  is:  Will  the  end  product  be  available  and  operate  when  and  for  as  long  as  needed?  Dependability 
can  be  a  function  of  the  system’s  ability  to  survive  in  the  environment  it  is  used;  its  vulnerability  to  external 
threats  such  as  misuse  by  operators,  destructive  forces  or  electromagnetic  environments;  aging  degradation 
(wear  out);  and  its  maintenance  status,  readiness  status,  usage  rates,  durability,  mobility,  flexibility,  and 
repairability;  or  a  failure  within  the  product  before  it  completes  its  task. 

Suitability  -  a  measure  of  the  degree  to  which  an  end  product  is  appropriate  for  its  intended  use.  The 
question  answered  is:  Is  it  the  right  end  product  for  the  task?  Suitability  involves  having  the  right  non- 
operational  attributes  designed  into  the  end  product  -  interoperability,  compatibility,  deployability, 
transportability,  usability,  supportability,  and  maintainability.  In  addition,  an  end  product  has  to  interface 
correctly  with  other  products,  with  operators  and  within  the  internal  and  external  operating  environments. 
Enabling  products  also  need  to  be  in  place  and  implemented  when  needed  by  the  operational  end  product. 
Enabling  products  include:  appropriate  training  curricula,  facilities  and  manuals;  packaging,  handling  and 
storage  provisions;  facilities  and  processes  for  proper  disposal  of  product  parts,  especially  hazardous 
materials  at  the  end  of  useful  life  of  a  product  and  hazardous  wastes  during  product  use;  and  the  production, 
test,  operational,  maintenance  and  support  facilities,  equipment,  tools,  and  manuals.  The  end  products  with 
these  sets  of  enabling  products  make  up  the  system  of  interest  for  a  system  effectiveness  analysis. 

Cost  effectiveness  -  a  measure  of  the  suitability,  dependability  and  capability  added  by  an  end  product  and 
its  enabling  products  as  a  function  of  total  ownership  costs.  The  question  answered  is:  Is  the  system 
affordable?  Several  total  ownership  cost  measures  are  used: 

•  Unit  cost  -  the  cost  of  the  delivered  end  product  as  a  function  of  management,  hardware,  software, 
non-recurring  start-up,  and  allowance  for  change  cost.  (Also  known  as  “flyaway  cost”). 

•  System  Cost  -  the  unit  cost  plus  technical  data,  publications,  contractor  services,  training  and  support 
equipment,  and  factory  training. 

•  Procurement  cost  -  the  system  cost  plus  the  cost  of  initial  spares. 

•  Acquisition  cost  -  the  procurement  cost  plus  RDT&E  and  facility  construction. 

•  Life  cycle  cost  -  the  acquisition  cost  plus  operations,  support  (including  post-production  support),  and 
disposal  costs. 

•  Total  ownership  cost  -  the  total  cost  to  the  owner  over  the  life  of  the  end  product.  This  includes  the 
procurement  cost  for  the  end  product  and  all  related  costs  thereafter  for  deploying,  training,  using, 
supporting,  maintaining,  and  disposing/retiring  the  end  product  as  well  as  any  associated  enabling 
products  needed  to  enable  the  end  product  to  meet  its  life  cycle  functionality.  It  is  equivalent  to  life 
cycle  cost. 
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•  Cost  Estimating  Relationships  (CER)  -  used  for  parametric  analysis  of  costs.  Allows  use  of  cost  data 
from  other  projects  for  similarity  analysis.  Examples  of  CER  factors  are  number  of  interfaces, 
complexity  of  interfaces,  type  of  interfaces,  platform,  reliability  levels,  and  support  factors  (local  vs. 
depot,  user  vs.  contractor).  (Part  of  parametric  cost  estimation  technique  used  during  early  phases  of 
development.) 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  are  used,  as  appropriate,  to:  (1)  assess  each  alternative  physical  solution 
representation;  (2)  assist  in  choosing  the  preferred  physical  solution  representation;  and  (3)  provide  the 
assessments  for  trade-off  analyses  to  aid  in  determining  recommended  decisions  and  their  effects. 


115 


Sub-process  22 


Sub-process  23  -  Trade-off  Analysis 

The  developer  shall  perform  Trade-off  analyses  to  provide  the  decision  makers  (i.e.,  Program 
Managers  and  Engineers)  with  recommendations,  predictions  of  the  results  of  alternative 
decisions,  and  other  appropriate  information  to  allow  selection  of  the  best  course  of  action. 


A  Trade-off  Analysis  may  be  required  at  any  phase  of  the  overall  systems  engineering  process  and  at  any  level 
within  any  phase.  For  example,  a  Trade-off  Analysis  may  involve  comparisons  of  air  platform  types,  system 
operational  concepts,  system  designs,  subsystem  designs,  or  component  selection. 


Types  of  Trade-off  processes  include  (but  are  not  limited  to): 


Trade-off  Analysis  Type 

Example  Description 

References 

Radar  System  AOA 

(Example  of  a  System  Performance 

&  Constraints  Trade-off  Report; 

relates  to  System  Technical 

Requirements) 

A  Trade-off  analysis  to  determine 
which  radar  system  will  best  meet 
Marine  Corps  requirements. 

AIR  4.10  Warfare  Analysis 
Department  ‘Analysis  of 

Alternatives’  Process  in  the  archive 
of  the  Research  and  Engineering 
Process  Website. 

Aircraft  Trade-off  Analysis 
(Example  of  a  Mission  & 

Operational  Trade-off  Report; 
relates  to  Customer  and  Stakeholder) 

A  Trade-off  analysis  to  determine 
which  type  of  aircraft  will  provide 
the  best  performance  for  a  particular 
set  of  navy  missions  (e.g.,  turboprop, 
turbofan,  or  reciprocating  engine 
powered) 

AIR  4.10  Warfare  Analysis 
Department  ‘Warfare  Analysis’ 
Process  in  the  archive  of  the 

Research  and  Engineering  Process 
Website. 

Selection  of  best  Contractor  Concept 
for  a  specific  purpose 
(Example  of  a  Functional  Solution 
Trade-off  Report;  relates  to  Logical 
Solutions) 

A  Trade-off  analysis  to  determine 
which  Contractor  Concept  provides 
the  best  Cost/Effectiveness  for  the 
Navy. 

AIR  4.10  Warfare  Analysis 
Department  ‘Source  Selection’ 

Process  in  the  archive  of  the 

Research  and  Engineering  Process 
Website. 

Sensor  Trade-off  Analysis 
(Example  of  a  Design  Synthesis 
Solutions  and  Technologies  Trade¬ 
off  Report;  relates  to  Physical 
Solutions) 

A  Trade-off  analysis  to  determine 
which  sensor  would  provide  the 
greatest  effectiveness  for  a  missile  of 
a  specific  design. 

AIR  4.10  Warfare  Analysis 
Department  ‘Analysis  of 

Alternatives’  Process  in  the  archive 
of  the  Research  and  Engineering 
Process  Website. 

Additional  information  on  performing  a  Trade-off  analysis  can  be  found  in  the  INCOSE  SE  Handbook,  Sections 
4.5.1  and  4.5.2. 


Preceding  Process 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Sub-process  10:  Progress  Against  Requirements 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  24:  Risk  Analysis 
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The  Trade-off  Analysis  process  may  be  invoked  by  any  of  these  processes  either  on  a  singular  or  multiple  bases, 
and  may  be  invoked  at  any  phase  of  the  overall  systems  engineering  process. 

Inputs 

•  Trade  options  and  constraints  (SP  16,  17,  18) 

•  Plans  and  schedules  trend  analysis  (SP  9) 

•  Requirement  trend  analysis  (SP  10) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Risk  Analysis  Report  (SP  24) 

The  trade  option  is  a  general  trade  problem  and/or  the  specific  alternatives  to  be  considered.  Constraints  are 
things  like  schedule  limitations,  cost  limitations,  and  organizations  to  be  involved. 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents.  For  example,  controversial  inputs/data  may 
be  required  to  perform  a  Trade-off  Analysis  for  competing  subsystems  within  a  larger  system.  Using  the  wrong 
data  in  that  case  may  not  only  lead  to  wrong  conclusions  and/or  an  inferior  subsystem/system  design,  but  may 
have  further  non-systems  engineering  ramifications  such  as  a  contractor  protest  or  a  legal  action  related  to  the 
erroneous  Trade-off  Analysis  results.  Before  beginning,  ensure  the  trade-off  problem  definition  is  complete. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  The  entire  study  (analysis) 
has  to  be  done  in  a  rigorous  and  defensible  manner  such  that  it  can  withstand  high  and  detailed  levels  of 
scrutiny.  Questions  and  points  of  contention  must  be  thought  out  beforehand.  Tasks  to  consider  include  (but 
are  not  limited  to)  the  following: 

a)  Plan  trade-off  analyses  and  develop  a  Trade-Off  Analysis  Plan  of  Actions  and  Milestones 

(POA&M)  to  include: 

•  the  availability  and  definition  of  required  resources,  execution  and  data  collection  requirements, 
expected  outcomes,  defined  conditions  (triggers  and  rigor),  level  of  importance,  objectives, 
schedule  of  tasks,  and  type  (formal,  informal,  judgmental;  see  Table  C.23,  Appendix  C) 

•  selection  criteria  that  will  determine  desirability  or  undesirability  of  an  option  or  alternative  for 
example,  cost,  schedule  performance  and  risk;  life-cycle  outcomes;  -ility  concerns  (e.g., 
producibility,  testability,  maintainability,  supportability,  and  disposability);  size,  weight,  and 
power  consumption;  and  effectiveness  analysis  outcomes 

•  weighting  factors  (if  applicable)  for  each  selection  criterion  in  order  to  distinguish  its  degree  of 
importance 

•  models  and  tools  (representative  or  simulation)  to  be  used  in  the  trade-off  analysis 

•  analysis  to  be  performed,  including  sensitivity  and  metrics  by  which  to  compare  alternatives 

•  options  or  alternatives  to  be  analyzed. 

b)  Perform  the  Trade-off  analysis  according  to  the  POA&M,  and: 

1)  Do  appropriate  effectiveness  analysis  tasks  (Sub-process  22)  to  provide  a  quantitative  basis  for 
evaluating  options. 

2)  Do  appropriate  risk  analysis  tasks  (Sub-process  24)  to  quantitatively  assess  the  risk  associated 
with  each  option. 

3)  Collect  data  and  analyze  it  to  determine  the  cost,  schedule,  performance,  and  risk  effect  of  each 
option  or  alternative. 

4)  Evaluate  options  against  selection  criteria  and  weighting  factors,  and  identify  and  define 
recommendations  (if  applicable).  Weighting  criteria  is  quantified  on  the  basis  of  the  relative 
importance  level  of  the  associated  attribute  (e.g.,  if  SPEED  is  twice  as  important  as  RANGE  then 
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the  SPEED  weighting  factor  might  be  2  and  the  RANGE  weighting  factor  might  be  1  or  multiples 
thereof);  often  weighting  factors  are  normalized,  (e.g.,  the  normalized  weighting  factors  are 
computer  as  follows:  normalized  weighting  factor(i)  =W(I),  and  W(I)=weighting_factor(i)/X  (all 
weighting  factors)).  Important  parameters,  relative  importance,  and  quantified  weights  can  be 
partially  or  wholly  through  use  of  precedent,  research,  testing,  expert  opinion,  the  Delphi 
technique,  and  other  methods  as  may  be  appropriate  to  the  Trade-off  Analysis  being  performed. 
Weights  can  also  be  applied  parametrically  such  that  variation  of  the  weighting  criteria  on  results 
can  be  studied.  Also,  sensitivity  of  a  Trade-off  Analysis  results  may  be  studied  by  varying  or 
parameterizing  the  appropriate  set  of  analysis  variables. 

5)  Produce  a  Trade-off  Analysis  Document  and  Trade-off  Study  Brief.  The  Trade-off  analysis 
documentation  includes,  at  minimum,  the  following: 

Tasking  and  Problem  Statement/Formulation 

Rationale  for  Study/ Analysis 

Scope  of  Study/ Analysis 

Trade-off  Analysis  Team  Description 

Schedule 

Choices  and  explanations 
Analysis  performed 
Weighting  Factors,  if  applicable 
Resulting  order  of  choices 
Rationale/Explanation  for  results 
Implications  of  each  choice 
Criteria  for  Choices 
Alternative-Criteria  Matrix 

The  Alternative-Criteria  Matrix  is  a  matrix  depicting  the  alternatives  (that  are  the 
subject  of  the  trade  study)  and  displaying  them  in  a  tabular  form  versus  the  criteria 
for  choices  to  be  used  in  the  Trade-off  Analysis.  E.g.; 


ALTERNATIVE 

CRITERIA 

RANGE 

PAYLOAD 

SORTIE  RATE 

SPEED 

HELICOPTER 

TURBOPROP 

TILT  ROTOR 

Sensitivities 

Utility  Curves,  if  applicable 

The  desirability  of  alternatives  can  be  measured  quantitatively  by  defining  utility 
functions.  Using  the  oversimplified  example  above,  such  a  utility  function  may  be 
U=PAYLOAD*SORTIE_RATE.  This  utility  function  would  provide  a  measure  of 
payload  delivery  capacity/time  period.  The  utility  function  can  be  computed  and 
plotted  for  each  alternative  to  produce  a  utility  curve. 

Conclusions 

Recommendations,  if  applicable 

Annexes  (for  applicable  required  and  detailed  data). 

6)  Communicate  recommendations  and  impacts  to  appropriate  decision  makers. 


c)  Record  the  outcomes  of  the  Trade-off  analysis  in  the  enterprise  data  repository,  including 

assumptions,  details  of  the  analysis,  lessons  learned,  models  used,  rationale  for  decisions  made, 
recommendations  and  effects,  and  other  pertinent  information  affecting  the  interpretation  of  the 
decision  made.  (Sub-process  12) 
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Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.  “EXT”  indicates 
it  is  external,  unspecified,  and  not  for  a  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Trade-off  Analysis  POA&M  (SP  23) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Risk  Analysis  Request  (SP  24) 

•  Trade-off  Analysis  Technical  Report  (SP  9,  16,  17,  18) 

•  Trade-off  Analysis  Presentation  (EXT) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Trade-off  study  is  complete. 

Results  are  archived. 

Next  Processes 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  24:  Risk  Analysis 

The  results  of  the  Trade-off  Analysis  will  be  provided  to  the  invoking  process,  archiving  processes,  and  other 
systems  engineering  processes  as  determined  and  deemed  appropriate  prior  to  study  start. 

Agents 

Program  Management 
System  Engineering 
Analysis 

Tools 

Analysis: 

•  Excel  with  VBA 

•  Access 

•  Visual  Basic,  C 

•  Warfare  &  System,  subsystem  models 

•  Integrated  architecture  products 

Planning/Documentation: 

•  Project 

•  Schedule 

•  Word 

•  PowerPoint 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 
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•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 


Naval  Operations  Analysis,  D  Wagner,  C  Mylander,  T  Sanders,  1999. 

Simulation  and  Modeling  Analysis,  M  Law,  D  Kelton,  1981&  1982. 

System  Engineering  Management,  James  Lacy,  1992. 

AIR  4.10  Warfare  Analysis  Department  ‘Analysis  of  Alternatives’  Process  in  the  archive  of  the  Research  and 
Engineering  Process  Website. 

AIR  4.10  Warfare  Analysis  Department  ‘Warfare  Analysis’  Process  in  the  archive  of  the  Research  and 
Engineering  Process  Website. 

AIR  4.10  Warfare  Analysis  Department  ‘Source  Selection  Process’  Process  in  the  archive  of  the  Research  and 
Engineering  Process  Website. 


Metrics  and  Measures 

Trade-off  study  completion  and  acceptance  by  the  appropriate  agent. 

Adherence  to  schedule. 

Adherence  to  funding  plan. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  are  used,  as  appropriate,  to  resolve  requirement  conflicts  during  requirements 
definition;  to  assess  groupings  of  functions,  objects,  etc.,  during  definition  of  logical  solution  representations;  to 
assess  design  options  and  alternative  physical  solution  representations  during  definition  of  physical  solution 
representations;  to  determine  progress  in  satisfying  technical  requirements;  and  to  evaluate  outcomes  of 
verifications  and  validations. 


Sub-process  24  -  Risk  Analysis 

The  developer  shall  perform  risk  analyses  to  develop  risk  management  strategies,  support 
management  of  risk,  and  support  decision-making. 


Preceding  Process 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Sub-process  7:  Technical  Plans 
Assessment  Process 

Sub-process  9:  Progress  Against  Plans  and  Schedules 
Sub-process  10:  Progress  Against  Requirements 
Solution  Definition  Process 

Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  22:  Effectiveness  Analysis 
Sub-process  23:  Trade-off  Analysis 

Inputs 

•  Risk  Management  Strategy  (including  Risk  Advisory  Board  requirements)  (SP  5) 

•  Risk  Management  Plan  (SP  7) 

•  System  Engineering  Plan  (SEP)  or  Software  Development  Plan  (SDP)  (SP  7) 

•  Plans  and  schedules  trend  analysis  (SP  9) 
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Requirements  trend  analysis  (SP  10) 
Risk  Analysis  Request  (SP  18,  23) 
Effectiveness  Analysis  Report  SP  22) 


The  above  input  techniques  define  product  characteristics,  V&V  results,  and  requirement  conflicts  and  issues. 
Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  appropriate  tasks  to  complete  this  sub-process.  These  efforts  seldom  include 
‘easy  answers,’  so  team  efforts  such  as  brainstorming  and  interviews  are  often  employed  in  this  process.  Tasks 
to  consider  include  the  following: 

a)  Identification  of  technical  risk,  and  resulting  project  risk,  based  on  exposure  to  the  probability  of 
an  undesireable  consequence  and  the  effect  of  that  consequence  for  each  Trade-off  analysis 
option  for  each  physical  solution  representation.  The  Representative  for  Engineering  (NAV AIR’s 
APMSE)  asks  other  agents  (see  list  later  in  this  requirement)  from  the  Research  and  Engineering 
Group  to  identify  technical  performance  parameters  that  the  system  must  meet.  He  in  turn  asks  those 
agents  to  identify  technical  risks.  These  risks  include  safety  items,  technical  performance  parameters 
that  the  system  may  fall  short  of  and  programatic  (cost  and  schedule)  constraints  that  pose  challenges 
to  the  program. 

Risk  Analysis:  Software  Safety.  Software  safety  risk  is  a  sub-discipline  of  system  safety.  Software  is 
deemed  safe  if  it  is  impossible  (or  at  least  highly  unlikely)  that  the  software  could  ever  produce  an 
output  that  would  cause  a  catastrophic  event  for  the  system  that  the  software  controls.  Catastrophic 
events  may  include  loss  of  physical  property,  physical  harm,  and  loss-of-life.  The  Software  Safety 
discipline  refers  to  a  broad  class  of  development  and  assessment  processes  that  attempt  to  make 
software  safe.  This  may  include  techniques  such  as  fault-tree  analysis  (FTA),  formal  methods 
(particularly  those  aimed  at  early  life-cycle  phases),  Petri  nets,  Failure  Modes  Effect  and  Criticality 
Analysis  (FMECA),  HAZOP,  and  impact  analysis. 

b)  Characterize  risks  by  causes,  possible  effects  or  consequences,  likelihood  of  occurrence,  options 
for  dealing  with  risks,  how  long  option  is  available,  and  coupling  with  other  risks.  It  is  usually 
imposible  to  quantify  the  consequence  and  likelihood  of  a  risk  related  to  a  new  system.  This  is  in 
sharp  contrast  to  the  insurance  industry  where  actuarials  precisely  quantify  both  parameters. 

Tools  used  to  get  quantative  estimates  are  schedule  network  models  (consequence),  reliability  models 
(likelihood),  sensitivity  analyses  (consequence)  and  technical  performance  tracking  tools  (likelihood). 

In  the  DAU  Program  Manager’s  Tool  Kit,  parameters  are  used  to  quantify  risk  based  upon  semi- 
quantative  decisions.  Particular  parameters  for  likelihood  and  for  consequence  are  presented.  When 
using  this  method,  tailor  their  parameters  for  the  specific  application.  When  using  these  criteria,  keep 
the  same  tailored  parameters  in  characterizing  every  risk  associated  with  the  system. 

An  alternative  to  DAU’s  semi-qualitative  method  is  a  qualitative  “Rubic’s  cube”  approach.  In  this 
approach,  the  risk  management  board,  with  inputs  from  the  appropriate  agents,  rates  the  consequence 
and  likelihood  of  each  risk  on  a  one-to-five  scale.  Again,  the  same  scale  must  be  used  for  all  risks  in 
the  system  we  are  characterizing.  Depending  upon  which  of  the  25  blocks  in  the  five-by-five 
consequence-likelihood  matrix  the  risk  falls,  it  is  high,  medium  or  low.  NAVAIRINST  5000.21  is  a 
good  reference  for  this  process. 

c)  Prioritize  risks  that  would  likely  cause  harm,  have  the  greatest  effect  on  the  system,  and  would 
require  attention  in  the  near  term.  In  terms  of  risks  that  would  likely  cause  harm,  the  prioritization 
follows  the  philosophy  of  NAVAIRINST  5000.21,  Program/Project  Risk  Management  and  the  Hazard 
Analysis  task  of  MIL-STD-882.  From  a  safety  standpoint,  this  is  paramount.  From  a  greatest  effect  on 
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the  system,  there  are  often  risks  beyond  safety  risks.  The  prioritization  of  program  risks  includes  cost, 
schedule,  and  technical  (safety  is  a  subset  of  technical)  risks.  This  process  includes  prioritizing  all  of 
the  risks  -  considering  both  likelihood  and  consequence.  The  risk  management  board  then  places  high 
and  medium  risks  on  a  watchlist  for  continued  surveillance. 

d)  Evaluate  ways  to  avert  risk,  and  determine  the  cost,  schedule,  and  performance  effects  on  the 
project. 

e)  Define  and  implement  a  plan  or  approach  for  averting  each  significant  risk. 

f)  Record  the  risk  analysis  outcomes  in  the  enterprise  data  repository  and  communicate  or  use  risk 
findings  and  impacts,  as  appropriate. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  List  of  risks  (SP  24) 

•  Analyses  of  Risk  Severity  (SP  24) 

•  Risk  Summary  Worksheet  (SP  24) 

•  Waterfall  Charts  (SP  24) 

•  Risk  Analysis  Report  (SP  18,  23) 

The  Risk  Analysis  Report  includes  information  such  as:  Lists  of  Risks,  Analyses  of  Risk  Severity,  Watch  Lists, 
Waterfall  Charts,  and  Risk  Summary  Worksheets.  The  last  is  a  risk  summary  displaying  all  significant  program 
risks  on  a  single  Analysis  of  Severity  Chart  (Rubio’ s  Cube),  and  is  the  output  NAVAIR  uses  the  most. 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  18:  Physical  Solution  Representations 
Systems  Analysis  Process 

Sub-process  23:  Trade-off  Analysis 

Agents 

Program  Manager,  Systems  Engineering,  Reliability  &  Maintainability,  Systems  Development  &  Integration, 
Weights,  Safety,  Software 

Typically  a  program  level  Risk  Management  Board  manages  the  risk.  That  board  is  comprised  of  Program 
Management  (1.0)  and  Systems  Engineering.  Systems  Engineering  includes: 

The  Representative  for  Engineering  (NAVAIR’ s  APMSE)  receives  technical  inputs  from  engineers 
throughout  the  Systems  Engineering  department  (e.g.,  Systems  Development  and  Integration,  Weights, 
Reliability  and  Maintainability,  Safety,  and  Software)  and  from  systems  engineers  in  the  systems 
engineering  divisions  throughout  the  Research  and  Engineering  Group.  For  contracted  acquisitions,  the 
Representative  for  Engineering  works  closely  with  the  chief  engineer,  the  systems  engineer,  and  the  prime 
contractor  to  identify,  assess  and  control  risk. 

Tools 

Program  Risk  Summaries  (“Rubic’s  cubes”) 

DSMC  “Weighted  Factors” 

Schedule  Network  Models 
R&M  Models 
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TPM  Tracking  tools 
Integrated  architecture  products 


References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Risk  Management  process  areas 

DAU  Program  Manager’s  Tool  Kit 

DAU  Systems  Engineering  Fundamentals 

NAY  AIR  INST  5000.21  Program/Project  Risk  Management 

NAVSOP-3686 

Top  Eleven  Ways  to  Manage  Technical  Risk 

MIL-STD-882 

Metrics  and  Measures 

•  Qualitative  Risk  Severity  (where  is  it  on  Rubic’s  cube) 

•  Quantitative  Risk  Factor  (DSMC  Factors) 

•  Analog  of  Nichols  Growth  Curve  (keeping  up  with  mitigation  plan)  (Availability,  Reliability,  Capability, 


etc.) 


The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  help  in  problem  prevention  (i.e.,  to  identify  the  degree  of  risk  associated  with 
recommended  decision  alternatives  and  to  support  the  risk  management  program). 


4.5.2  Requirements  Validation  Process 

Requirements  Validation  is  critical  to  successful  system  product  development  and  implementation. 
Requirements  are  validated  when  it  is  certain  that  the  subject  set  of  requirements  describes  the  input 
requirements  and  objectives  such  that  the  resulting  system  products  can  satisfy  the  requirements  and  objectives. 
The  Requirements  Validation  Process  helps  ensure  that  the  requirements  are  necessary  and  sufficient  for 
creating  design  solutions  appropriate  to  meeting  the  exit  criteria  of  the  applicable  engineering  life-cycle  phase 
and  of  the  enterprise-based  life-cycle  phase  in  which  the  engineering  or  reengineering  efforts  occur. 

The  five  sub-processes  associated  with  the  Requirements  Validation  Process  are  shown  in  Figure  4.5.2a. 


Requirements 

Validation 

Process 

Requirements 


'Sub-process  27  -  Other  Stakeholder  Requirements  Validation 


-Sub-process  28  -  System  Technical  Requirements  Validation 


ub-process  25  -  Requirements  Statements  Validation 


ub-process  26  -  Acquirer  Requirements  Validation 


Sub-process  29  -  Logical  Solution  Representation  Validation 


Figure  4.5.2a  -  Requirements  Validation  Process/Sub-processes 
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One  or  more  of  these  five  sub-processes  are  invoked  by  a  recommended  task  within  either  the  Requirements 
Definition  Process  or  the  Solution  Definition  Process.  The  relationship  of  the  Requirements  Validation  Sub¬ 
processes  is  shown  in  Figure  4.5.2b. 


Figure  4.5.2b  -  Relationship  of  Requirements  Validation  Sub-processes 

This  Guide  separates  requirement  validation  into  two  areas.  The  first  area  is  the  statement  validation  and  the 
second  is  the  requirement  validation.  Sub-processes  16  through  19  define  contractual  requirement  statements 
(acquirer,  other  stakeholder,  or  derived),  therefore  they  all  call  out  Sub-process  25  to  validate  the  statements 
(ensuring  that  they  are  stating  the  appropriate  intent).  Sub-processes  14  and  15  do  not  produce  requirement 
statements  to  be  imposed  on  contract  and  therefore  the  statements  are  not  so  closely  scrutinized.  Sub-processes 
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14,  15,  16,  17  call  out  Sub-processes  26,  27,  28,  and  29,  respectively,  to  validate  the  requirement  that  is  being 
defined  for  the  system.  Sub-process  19  does  not  have  a  requirement  validation  process  because  it  is  a 
derivative  of  the  other  sub-processes. 

Requirements  should  be  validated  at  each  level  of  the  system  structure  for  requirements  definition.  Generally, 
validation  of  requirements  at  higher  levels  is  a  basis  for  validation  at  lower  levels  (see  Section  6). 


Sub-process  25  -  Requirement  Statements  Validation 

The  developer  shall  ensure  that  technical  requirements  statements  and  specified  requirements 
statements,  individually  and  as  sets,  are  well  formulated.  This  is  validation  of  the  language 
of  the  statements  rather  than  the  content. 


Preceding  Process 
Planning  Process 

Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:  Physical  Solution  Representations 
Sub-process  19:  Specified  Requirements 

Inputs 

•  Verification  Plan  (including  verification  matrix)  (SP  7) 

•  Validation  Plan  (SP  7) 

•  System  technical  requirements  (SP  1 6) 

•  Derived  technical  requirements  (SP  17,  18) 

•  Specified  requirements  (SP  19) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  The  tasks  below  are 
broken  down  into  task  a)  which  will  be  accomplished  on  each  requirement  statement  individually;  and  task  b) 
which  involves  looking  at  the  requirement  statements  in  various  combinations  and  then  as  a  whole.  Tasks  to 
consider  include  the  following: 

a)  Analyze  each  requirement  statement  from  Sub-processes  16,  17,  18,  and  J_9  to  ensure: 

1)  ability  to  preserve  competitiveness  -  permits  preservation  of  a  competitive  stance  and  is  only  as 
constraining  on  competitive  stance  as  is  justified  by  benefits  delivered  by  requirement. 

2)  clarity  -  requirement  statement  is  readily  understandable  without  analysis  of  meaning  of  words  or 
terms  used. 

3)  correctness  -  requirement  statement  does  not  contain  an  error  of  fact. 

4)  feasibility  -  requirement  can  be  satisfied  within  (1)  natural  physical  constraints,  (2)  state  of  the  art 
as  it  applies  to  the  project,  and  (3)  all  other  absolute  constraints  applying  to  the  project. 

5)  focus  -  requirement  is  expressed  in  terms  of  ‘what’  and  ‘why,’  or  form,  fit  and  function,  not  in 
terms  of  how  to  develop  the  products  or  the  materials  to  be  used  -  detailed  requirements  that  are 
required  to  guide  detailed  design  of  a  product  are  an  exception  to  this. 

6)  implementability  -  requirement  statement  contains  information  necessary  to  enable  requirement 
to  be  implemented. 

7)  modifiability  -  necessary  changes  to  a  requirement  can  be  made  completely  and  consistently. 
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8)  removal  of  ambiguity  -  allows  only  one  interpretation  for  meaning  of  the  requirement  (e.g.,  not 
defined  by  words  or  terms  such  as  ‘excessive,’  ‘sufficient,’  and  ‘resistant’  that  cannot  be 
measured). 

9)  singularity  -  requirement  statement  cannot  be  sensibly  expressed  as  two  or  more  requirements 
having  different  agents,  actions,  objects,  or  instruments. 

10)  testability  -  existence  of  finite  and  objective  process  with  which  to  verify  that  the  requirement  has 
been  satisfied. 

11)  verifiability  -  can  be  verified  at  the  level  of  system  structure  at  which  it  is  stated. 

and 

12)  performance  based  language  (where  appropriate)  -  requirement  statements  cannot  give 
direction  on  “how  to”  implement  a  specific  requirement.  They  need  to  indicate  only  the 
performance  and  boundary  conditions  of  the  requirement. 

b)  Analyze  requirement  statements  from  Sub-processes  16,  17,  18,  and  19  in  pairs  and  sets  to  ensure: 

1)  absence  of  redundancy  -  each  requirement  is  specified  only  once. 

2)  connectivity  -  all  terms  within  a  requirement  are  adequately  linked  to  other  requirements  and  to 
work  and  term  definitions,  so  that  individual  requirements  relate  properly  to  other  requirements  as 
a  set. 

and 

3)  removal  of  conflicts  -  requirement  is  not  in  conflict  with  other  requirements  or  within  itself. 

c)  Record  requirement  statements  validation  outcomes  in  the  established  enterprise  data 
repository. 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Requirement  statements  validation  revisions  (SP  16,  17,  18,  19) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent.  (Acceptable  sets  of  requirements 
statements) 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 
Sub-process  18:  Physical  Solution  Representations 
Sub-process  19:  Specified  Requirements 

Agents 

Systems  Engineering 
Technical  Writer 

Tools 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™). 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  C4ISR  Architecture  Framework 
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•  Joint  Technical  Architecture 

•  FAR/DFARs 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 
MIL-STD-961D 

SD-24:  General  Specification  Performance,  Design,  Characteristics,  and  Construction  of  Aircraft  Weapons 
Systems 

Joint  Services  Specification  Guides  (JSSG) 

Metrics  and  Measures 

Percentage  of  validated  requirements  statements 
Percentage  of  requirement  statements  issues 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  validated  technical 
requirement  statements  resulting  from  satisfying  this  sub-process  are  used  to  guide  development  of  system 
design  solutions  and  evolve  into  related  specified  requirements. 


Sub-process  26  -  Acquirer  Requirements  Validation 

The  developer  shall  ensure  that  the  set  of  defined  acquirer  requirements  agrees  with  acquirer 
needs  and  expectations. 


Preceding  Process 

Planning  Process 

Sub-process  7:  Technical  Plans 

Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 

Inputs 

•  Validation  Plan  (SP  7) 

•  Acquirer  Requirements  (SP  14) 

Entry  Criteria 

Inputs  have  been  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  The  tasks  of  this  sub¬ 
process  are  completed  to  ensure  both  the  correctness  and  traceability  of  the  Acquirer  Requirements.  Tasks  to 

consider  include  the  following: 

a)  Select  the  methods  and  define  the  procedures  for  validating  that  the  set  of  acquirer  requirements 
from  Sub-process  14  is  consistent  with  the  level  of  system  structure,  enterprise-based  life-cycle 
phase,  and  Validation  Plan,  as  appropriate.  The  method  may  be  via  computer  or  by  hand,  and  may 
incorporate  integrated  architecture  products,  spreadsheets  and/or  databases. 

Consideration  should  be  given  regarding  the  use  of  software  (such  as  CORE™  and  SLATE™) 
applications  for  requirements  management  and  integrated  architecture  development  to  provide  a  means 
to  trace  from  requirements  to  mission,  task  and  operational  activities  to  system  functions,  to  systems, 
and  then  on  to  technical  standards.  This  provides  the  abililty  to  assess  the  impact  on  operations  of 
shortfalls  in  sytstems  functions,  performance,  interface,  data,  installation,  or  standards. 

b)  Analyze  and  compare  the  identified,  derived,  and  collected  acquirer  requirements  to  the  set  of 
defined  acquirer  requirements,  to  determine  downward  traceability.  The  methods  and  procedures 
selected  in  task  a)  above  should  be  applied  to  create  a  traceabilty  matrix.  This  information  is  an 


127 


Sub-process  26 


automatically  generated  output  of  many  of  the  requirements  management  and  system  architecture 
software  applications. 

c)  Analyze  and  compare  the  set  of  defined  acquirer  requirements  to  the  identified,  derived,  and 
collected  acquirer  requirements,  to  determine  upward  traceability.  The  methods  and  procedures 
selected  in  task  a)  above  should  be  applied  to  create  a  traceabilty  matrix.  This  information  is  an 
automatically  generated  output  of  many  of  the  requirements  management  and  system  architecture 
software  applications. 

d)  Identify  and  resolve  variances,  voids,  and  conflicts  (orphans).  Return  to  Sub-process  14  to 

produce  more  appropriate  Acquirer  Requirements. 

e)  Record  validation  results  in  the  established  enterprise  data  repository. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Validation  methods  &  procedures  (SP  26) 

•  Requirements  traceability  matrix  (SP  26) 

•  Acquirer  requirements  validation  revisions  (SP  14) 

Exit  Criteria 

Outputs  have  been  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 

Agents 

Systems  Engineering 

R&M 

Safety 

Supportability/T  estability 
Tools 

Requirements  Traceability  Matrix  Format 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

Metrics  and  Measures 

Percent  of  Acquirer  Requirements  downward  traceable 
Percent  Acquirer  Requirements  upward  traceable 

Percent  of  assumptions  for  Acquirer  Requirements  reviewed  and  approved 
Percent  of  changed  Acquirer  Requirements  revalidated 
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The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process,  when  combined  with  other  stakeholder  requirements,  provide  inputs  to  the 
definition  of  system  technical  requirements  (see  Appendix  F). 


Sub-process  27  -  Other  Stakeholder  Requirements  Validation 

The  developer  shall  ensure  that  the  set  of  defined  other  stakeholder  requirements  agrees  with 
other  stakeholder  needs  and  expectations  with  respect  to  the  system. 


Preceding  Process 
Planning  Process 

Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  15:  Other  Stakeholder  Requirements 

Inputs 

•  Validation  Plan  (SP  7) 

•  Other  Stakeholder  Requirements  (SP  15) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  The  tasks  of  this  sub¬ 
process  are  completed  to  ensure  both  the  correctness  and  traceability  of  Other  Stakeholder  Requirements.  Tasks 
to  consider  include  the  following: 

a)  Select  the  methods  and  define  the  procedures  for  validating  that  the  set  of  other  stakeholder 
requirements  from  Sub-process  15  is  consistent  with  the  level  of  system  structure,  enterprise- 
based  life-cycle  phase,  and  Validation  Plan,  as  appropriate.  The  method  may  be  via  computer  or 
by  hand,  and  may  incorporate  integrated  architecture  products,  spreadsheets  and/or  databases. 

Consideration  should  be  given  regarding  the  use  of  software  (such  as  CORE™  and  SLATE™) 
applications  for  requirements  management  and  integrated  architecture  development  to  provide  a  means 
to  trace  from  requirements  to  mission,  task  and  operational  activities  to  system  functions,  to  systems, 
and  then  on  to  technical  standards.  This  provides  the  abililty  to  assess  the  impact  on  operations  of 
shortfalls  in  system  functions,  performance,  interface,  data,  installation,  or  standards. 

b)  Analyze  and  compare  the  identified,  derived,  and  collected  other  stakeholder  requirements  to  the 
set  of  defined  other  stakeholder  requirements,  to  determine  downward  traceability.  The  methods 
and  procedures  selected  in  task  a)  above  should  be  applied  to  create  a  traceabilty  matrix.  This 
information  is  an  automatically  generated  output  of  many  of  the  requirements  management  and  system 
architecture  software  applications. 

c)  Analyze  and  compare  the  set  of  defined  other  stakeholder  requirements  to  the  identified, 
derived,  and  collected  other  stakeholder  requirements,  to  deterimine  upward  traceability.  The 

methods  and  procedures  selected  in  task  a)  above  should  be  applied  to  create  a  traceabilty  matrix.  This 
information  is  an  automatically  generated  output  of  many  of  the  requirements  management  and  system 
architecture  software  applications. 

d)  Identify  and  resolve  variances,  voids,  and  conflicts  (orphans).  Return  to  Sub-process  15  to 

produce  more  appropriate  Other  Stakeholder  Requirements. 
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e)  Record  validation  results  in  the  established  enterprise  data  repository. 


Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Validation  methods  &  procedures  (SP  27) 

•  Requirements  Traceability  Matrix  (SP  27) 

•  Other  stakeholder  requirements  validation  revisions  (SP  15) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Requirements  Definition  Process 

Sub-process  15:  Other  Stakeholder  Requirements 

Agents 

Systems  Engineering 

R&M 

Safety 

Supportability/T  estability 
Tools 

Requirements  Traceability  Matrix  Format 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

Metrics  and  Measures 

Percent  of  Other  Stakeholder  Requirements  downward  traceable 
Percent  Other  Stakeholder  Requirements  upward  traceable 

Percent  of  assumptions  for  Other  Stakeholder  Requirements  reviewed  and  approved 
Percent  of  changed  Other  Stakeholder  Requirements  revalidated 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process,  when  combined  with  acquirer  requirements,  provide  inputs  for  defining  the 
system  technical  requirements  (see  Appendix  F). 
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Sub-process  28  -  System  Technical  Requirements  Validation 

The  developer  shall  ensure  that  the  set  of  defined  system  technical  requirements  agrees  with 
validated  acquirer  and  other  stakeholder  requirements. 


A  primary  intent  is  to  gage  the  Quality  Assurance  of  input  received  from  other  Requirements.  Quality 
Assurance  is  achieved  through  accounting  (e.g.,  requirements  tracing),  confirming  previous  assumptions,  and 
ascertaining  that  all  life  cycle  aspects  have  been  covered. 

Preceding  Process 
Planning  Process 

Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 

Inputs 

•  Validation  Plan  (SP  7) 

•  System  Technical  Requirements  (SP  16)  (including  Design  Information  and  ICD/CDD  revisions) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  The  tasks  of  this  sub¬ 
process  are  completed  to  ensure  both  the  correctness  and  traceability  of  System  Technical  Requirements.  Tasks 
to  consider  include  the  following: 

a)  Select  the  methods  and  define  the  procedures  for  validating  that  the  set  of  system  technical 
requirements  from  Sub-process  16  is  consistent  with  the  level  of  system  structure,  enterprise- 
based  life-cycle  phase,  and  Validation  Plan  (plan  content  to  be  determined)  as  appropriate.  The 
accounting  method  may  be  via  computer  or  by  hand,  and  may  incorporate  integrated  architecture 
products,  spreadsheets  and/or  databases. 

Consideration  should  be  given  regarding  the  use  of  software  (such  as  CORE™  and  SLATE™) 
applications  for  requirements  management  and  integrated  architecture  development  to  provide  a  means 
to  trace  from  requirements  to  mission,  task  and  operational  activities  to  system  functions,  to  systems, 
and  then  on  to  technical  standards.  This  provides  the  abililty  to  assess  the  impact  on  operations  of 
shortfalls  in  sytstems  functions,  performance,  interface,  data,  installation,  or  standards.  Performance  of 
the  other  tasks  should  include  customer  participation,  and,  if  appropriate,  an  independent  review. 

b)  Analyze  and  compare  the  set  of  validated  acquirer  and  other  stakeholder  requirements  to  the  set 
of  defined  system  technical  requirements,  to  determine  downward  traceability.  The  methods  and 
procedures  selected  in  task  a)  above  should  be  applied  to  create  a  traceabilty  matrix.  This  information 
is  an  automatically  generated  output  of  many  of  the  requirements  management  and  system  architecture 
software  applications. 

c)  Analyze  and  compare  the  set  of  defined  system  techncial  requirements  to  the  validated  set  of 
acquirer  and  other  stakeholder  requirements,  to  determine  upward  traceability.  The  methods 
and  procedures  selected  in  task  a)  above  should  be  applied  to  create  a  traceabilty  matrix.  This 
information  is  an  automatically  generated  output  of  many  of  the  requirements  management  and  system 
architecture  software  applications. 

d)  Analyze  assumptions  made  with  respect  to  defining  system  technical  requirements  to  ensure  that 
they  are  consistent  with  the  system  being  engineered.  Review  key  drivers  (e.g.,  MOE  or  design 
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constraints)  with  the  customer  to  confirm  consistency  with  current  objectives  and  development 
approach. 

e)  Analyze  system  technical  requirements  that  have  been  defined  as  essential  for  the  design  effort 
for  other  life-cycle  considerations  for  which  there  is  no  parent  requirement  in  the  set  of  acquirer 
and  other  stakeholder  requirements,  to  ensure  that  they  are  consistent  with  the  system  being 
engineered  and  other  system  technical  requirements.  Examples  of  life-cycle  activities  for  which  parent 
requirements  often  do  not  exist  are  manufacturing,  maintenance,  training,  or  disposal.  This  task  is  to 
ascertain  that  all  life-cycle  aspects  of  the  product  have  been  considered  and  that  associated 
requirements  are  defined. 

f)  Identify  and  resolve  variances,  voids,  and  conflicts  (e.g.,  omissions  and  orphans).  Return  to  Sub¬ 
process  16  to  produce  more  appropriate  System  Technical  Requirements. 

g)  Revalidate  the  system  technical  requirements  whenever  a  requirement  change  is  made  that 
affects  the  acquirer  requirements,  other  stakeholder  requirements,  or  system  technical 
requirements. 

h)  Record  validation  results,  including  lessons  learned,  in  the  established  enterprise  data  repository. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Validation  Methods  &  Procedures  (SP  28) 

•  Requirements  Traceability  Matrix  (SP  28) 

•  System  technical  requirements  validation  revisions  (SP  1 6) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 

Control  Process 

Sub-process  12:  Outcomes  Management 

Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 


Agents 

Systems  Engineering 

R&M 

Safety 

Supportability/T  estability 
Tools 

Requirements  Traceability  Matrix  Format 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 
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Metrics  and  Measures 

Percent  of  System  Technical  Requirements  downward  traceable 

Percent  of  System  Technical  Requirements  upward  traceable 

Percent  of  assumptions  for  System  Technical  Requirements  reviewed  and  approved 

Percent  of  changed  System  Technical  Requirements  revalidated 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  show  that  the  set  of  system  technical  requirements  has  traceability  from  the  set 
of  validated  stakeholders’  requirements  that  it  is  both  necessary  and  sufficient  as  inputs  for  the  definition  of 
logical  solution  representations  (see  Appendix  FT 


Sub-process  29  -  Logical  Solution  Representations  Validation 

The  developer  shall  ensure  that  the  set  of  logical  solution  representations  agrees  with  the 
appropriately  assigned  subset  of  system  technical  requirements. 


Preceding  Process 

Planning  Process 

Sub-process  7:  Technical  Plans 

Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 

Inputs 

•  Validation  Plan  (SP  7) 

•  Logical  Solution  Representation  (SP  17) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  The  tasks  of  this  sub¬ 
process  are  completed  to  ensure  both  the  correctness  and  traceability  of  the  Logical  Solution  Representation. 

Tasks  to  consider  include  the  following: 

a)  Select  the  methods  and  define  the  procedures  for  validating  that  the  sets  of  logical  solution 
representations  and  derived  technical  requirements  from  Sub-process  17  are  consistent  with  the 
level  of  system  structure,  enterprise-based  life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

The  method  may  be  via  computer  or  by  hand,  and  may  incorporate  integrated  architecture  products, 
spreadsheets  and/or  databases. 

Consideration  should  be  given  regarding  the  use  of  software  (such  as  CORE™  and  SLATE™) 
applications  for  requirements  management  and  integrated  architecture  development  to  provide  a  means 
to  trace  from  requirements  to  mission,  task  and  operational  activities  to  system  functions,  to  systems, 
and  then  on  to  technical  standards.  This  provides  the  abililty  to  assess  the  impact  on  operations  of 
shortfalls  in  sytstems  functions,  performance,  interface,  data,  installation,  or  standards. 

b)  Analyze  and  compare  the  set  of  validated  system  technical  requirements  to  the  set  of  defined 
logical  solution  representations  and  derived  techical  requirements,  to  determine  downward 
traceability.  The  methods  and  procedures  selected  in  task  a)  above  should  be  applied  to  create  a 
traceabilty  matrix.  This  information  is  an  automatically  generated  output  of  many  of  the  requirements 
management  and  system  architecture  software  applications. 

c)  Analyze  and  compare  the  set  of  defined  logical  solution  representations,  derived  technical 
requirements,  and  any  unassigned  system  technical  requirements  (see  the  note  under  Sub-process 
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17,  task  c)  to  the  validated  set  of  system  technical  requirements,  to  determine  upward 
treaceability.  The  methods  and  procedures  selected  in  task  a)  above  should  be  applied  to  create  a 
traceabilty  matrix.  This  information  is  an  automatically  generated  output  of  many  of  the  requirements 
management  and  system  architecture  software  applications. 

d)  Analyze  assumptions  made  with  respect  to  defining  sets  of  logical  solution  representations  and 
derived  technical  requirements  to  ensure  that  they  are  consistent  with  the  system  technical 
requirements  and  the  system  being  engineered.  Accomplishing  this  sub-process  is  simply  ensuring 
the  System  Analysis  Process  (Sub-processes  22,  23,  24)  has  been  completed. 

e)  Identify  and  resolve  variances,  voids,  and  conflicts  (orphans).  Return  to  Sub-process  17  to 

produce  more  appropriate  Logical  Solution  Representations. 

f)  Revalidate  the  sets  of  logical  solution  representations  whenever  a  requirement  change  is  made 
that  affects  the  acquirer  requirements,  other  stakeholder  requirements,  system  technical 
requirements,  or  sets  of  defined  logical  solution  representations  and  derived  technical 
requirements. 

g)  Record  validation  results,  including  lessons  learned,  in  the  established  enterprise  data  repository. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Validation  Methods  &  Procedures  (SP  29) 

•  Requirements  Traceability  Matrix  (SP  29) 

•  Logical  Solution  Representation  validation  revisions  (SP  17) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 

Control  Process 

Sub-process  12:  Outcomes  Management 

Solution  Definition  Process 

Sub-process  17:  Logical  Solution  Representations 

Agents 

Systems  Engineering 

R&M 

Safety 

Supportability/T  estability 

Tools 

Requirements  Traceability  Matrix  Format 

Requirements  Management  &  System  Architecture  Database  (ex.  CORE™,  DOORS,  SLATE™) 

References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 
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Metrics  and  Measures 

Percent  of  Logical  Solution  Representations  downward  traceable 

Percent  of  Logical  Solution  Representations  upward  traceable 

Percent  of  assumptions  for  Logical  Solution  Representations  reviewed  and  approved 

Percent  of  changed  Logical  Solution  Representation  revalidated 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  derived  technical  requirements  and  logical  solution  representations  as 
inputs  into  the  definition  of  physical  solution  representations  (see  Appendix  F). 


4.5.3  System  Verification  Process 

The  System  Verification  Process  is  used  to  ascertain  that:  (1)  the  system  design  solution  generated  by 
implementing  Sub-process  19  is  consistent  with  its  source  requirements  (selected  preferred  physical  solution 
representation);  (2)  end  products  at  each  level  of  the  system  structure  implementation,  from  the  bottom  up,  (see 
Section  6)  meet  their  specified  requirements;  (3)  enabling  product  development  or  procurement  for  each 
associated  process  is  properly  progressing;  and  (4)  required  enabling  products  will  be  ready  and  available  when 
needed  to  perform. 


NOTE  -  Verification  consists  of  inspection,  reviews,  analyses,  demonstrations,  tests,  or  service 
experience  applied  in  accordance  with  the  verification  plan. 


The  three  sub-processes  associated  with  the  System  Verification  Process  are  shown  in  Figure  4.5.3. 

Sub-process  30  -  Design  Solution  Verification 

Sub-process  31  -  End  Products  Verification 

•Sub-process  32  -  Enabling  Products  Readiness 
Figure  4.5.3  -  System  Verification  Process/Sub-processes 


System 

Verification 

Process 

Requirements 


Sub-process  30  -  Design  Solution  Verification 

The  developer  shall  verify  that  each  end  product  defined  by  the  system  design  solution 
conforms  to  the  requirements  of  the  selected  physical  solution  representation  for  Hardware 
and  Software  (if  applicable). 


Design  Solution  Verification  methods  include  inspection,  analysis,  simulation,  demonstration  or  test  of 
prototypes,  mockups,  physical  models,  breadboards,  brassboards,  etc. 

Preceding  Process 
Planning  Process 

Sub-process  7:  Technical  Plans,  i.e.,  the  Verification  and  Validation  Plans 
Sub-process  8:  Work  Directives 
Requirements  Definition  Process 

Sub-process  16:  System  Technical  Requirements 
Solution  Definition  Process 

Sub-process  18:  Physical  Solution  Representations 
Sub-process  19:  Specified  Requirements 


135 


Sub-process  30 


Inputs 

•  Verification  Plan  (SP  7),  including  the  Verification  Compliance  Requirement  Matrix  (VCRM) 

•  System  Engineering  Plan  (SEP)  and/or  Software  Development  Plan  (SDP)  (SP  7) 

•  Test  and  Evaluation  Master  Plan  (TEMP)  (SP  7) 

•  Independent  Verification  and  Validation  (IV&V)  Plan  (SP  7) 

•  Team  Work  Plan  (TWP)  (SP  8) 

•  Statement  of  Objectives  (SOO)  (SP  8) 

•  Statement  of  Work  (SOW)  (SP  8) 

•  System  Technical  Requirements  (SP  16) 

•  Preferred  physical  solution  representation  (SP  1 8) 

•  Specified  Requirements  (SP  19) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include 
the  following: 

a)  Plan  the  design  solution  verification  in  accordance  with  the  Verification  Plan,  agreement, 
applicable  enterprise-based  life-cycle  phase,  and  to  the  level  in  the  system  structure.  The 

appropriate  level  could  vary  from  a  system  and  sub-system  down  to  the  component  level,  and  shall 
include: 

•  selection  and  definition  of  the  appropriate  method  for  design  solution  verification  (which  should 
come  from  a  detailed  test  plan  (SP  7)  that  describes  the  methods  and  processes  to  be  used  in 
verifying  compliance  against  the  specified  requirements); 

•  verification  procedures  to  be  followed  for  the  method  selected  (including  the  purpose  and 
objective  of  each  procedure,  pretest  action,  and  post-test  action;  and  the  criteria  for  determining 
the  success  or  failure  of  the  procedure);  and 

•  establishment  and  checkout  (for  example,  adequacy  and  completeness)  of  the  environment  (for 
example,  climatic  conditions,  equipment,  facilities,  and  measuring  devices,  etc.)  in  which  the 
verification  method  and  procedures  will  be  implemented. 

b)  Perform  the  planned  design  solution  verification  using  the  selection  methods  and  procedures 
within  the  established  verification  environment,  to: 

1)  collect  and  evaluate  verification  outcomes  to  either  show  conformance  to  the  requirements  of  the 
selected  physical  solution  representation  or  to  identify  variances  (unverifiable  requirements  and 
constraints);  and 

2)  resolve  variances,  as  appropriate,  and  re-verify  to  establish  compliance  when  the  cause  of  the 
variance  was  failure  to  properly  complete  the  fully  characterized  design 

Any  system  requirements  that  are  not  controllable  and  observable  shall  be  reported  as  an  unverifiable 
requirement  to  Sub-process  16  via  Sub-process  25,  but  should  be  confirmed  as  part  of  task  b)  1) 
above  as  well.  Variances  shall  be  documented  in  the  Design  Solution  Discrepancy  Reports  and/or 
integrated  enterprise  data  respository  for  evaluation  and  resolution. 

c)  Reverify  according  to  a  redesign  verification  plan,  test  method,  or  procedure  when  variances 
were  determined  to  be  caused  by  poor  verification  or  inadequate  verification  environmental 
preparation.  The  level  of  regression  testing  shall  depend  on  the  complexity  of  the  design  fix  and  the 
level  necessary  to  ensure  that  the  redesign  has  resolved  the  non-conformance,  and  been  re-addressed  in 
the  Test  Plan  (reference  first  bullet  of  task  a)  above). 

d)  Record  verification  results,  including:  corrective  actions  taken;  lessons  learned;  outcomes 
achieved;  Trade-off,  effectiveness,  and  risk  analyses  completed  with  resulting  key  decisions;  test 
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activities  completed;  variances;  and  the  verified  design  solution  in  the  established  enterprise  data 
repository.  Results  should  be  included  in  the  Redesigned  Verification  Plan,  and  shall  be  an  output  to 
Sub-process  31  (End  Product  Verification)  so  that  the  information  can  be  included  in  the  System 
Verification  process,  and  to  the  established  enterprise  data  respository  (Sub-process  12). 

Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Demonstration  Test  Readiness  Report  (DTRR)  (SP  12,  31) 

•  Design  solution  verification  report  (SP  31) 

•  Design  solution  deficiency  and  discrepancy  reports  (hardware  and  software,  if  applicable),  (SP  10,  11,  12, 
19,31). 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Assessment  Process 

Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
System  Verification  Process 

Sub-process  31:  End  Product  Verification 

Agents 

Manufacturing  should  identify  the  approach  for  duplicating  a  product  configuration  in  a  cost  effective  manner. 
The  qualification  of  the  manufacturing  process  must  ensure  the  adequacy  of  the  production  planning,  tool 
design,  and  assembly  methods.  Configuration  Control  should  be  established  to  ensure  that  both  the  production 
baseline  and  the  production  process  are  controlled  and  disciplined. 

Logistics  should  include  a  Part  Control  Plan,  which  provides  device  control  with  an  adequate  program  set  up 
with  vendors  to  ensure  adequate  controls.  Early  detection  of  parts  problems  is  a  key  to  a  low-risk  transition  to 
production.  The  consideration  of  spares  availability  for  the  operational  phase  should  impact  system  design 
during  the  development  phase. 

Product  Assurance  -  Specialty  Engineering: 

-  Producibility  measures  the  relative  ease  of  manufacturing  a  product.  Manufacturing  Plans  should  be 
reviewed  to  ensure  that  the  product  does  not  contain  any  high  risk  processes  and  that  the  risks  are  identified 
and  understood. 

-  Quality  Assurance  is  more  than  just  establishing  a  good  quality  inspection  system.  A  management 
commitment  to  defect  prevention  is  the  prime  ingredient  of  a  sound  defect  control  program.  A  good 
Quality  Assurance  program  ensures  that  all  Program  requirements  are  satisfied. 

-  Survivability  is  a  critical  part  of  the  design  process,  which  means  that  the  system  shall  be  survivable  to  the 
threat  levels  anticipated  in  their  operating  environment.  System  threats  shall  be  considered  and  fully 
assessed  as  early  as  possible  in  the  program,  usually  during  System  Development  and  Demonstration. 

-  Reliability  should  have  advanced  that  the  predicted  MTBF  (Mean  Time  Between  Failures)  is  at  least  1 .25 
times  the  required  MTBF.  Growth  slopes  and  assigned  risk  should  be  integrated  into  the  analysis.  +/-3dB 
is  the  typical  required  margin  (.707  or  1.414)  depending  upon  the  parameter  measured;  if  the  system 
developer  cannot  meet  or  exceed  this  requirement,  an  analysis  demonstrating  why  a  design  margin  cannot 
be  met  shall  be  provided. 

-  Maintainability  and  Supportability.  Maintainability  is  the  measure  of  the  ability  of  an  item  to  be  retained  or 
restored  to  a  specified  condition  or  maintainability  can  refer  to  the  ease  of  repair  and  replacement. 
Supportability  refers  to  the  ease  of  obtaining  spare  parts,  having  trained  personnel  and  the  ease  of  testing 
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the  system  being  supported.  Determinations  should  be  based  on  operational  requirements  and  life-cycle 
cost  considerations. 


Software  Development  shall  include  a  formalized,  intensive  design  effort  including  verification  and  validation 
of  the  requirements,  test  plans,  and  coding.  Integrated  software/hardware  systems  shall  be  tested  exhaustively 
in  a  total  system  test  bed. 

Systems  Engineering  shall  ensure  that  a  process  is  used  to  translate  operational  needs  and/or  requirements  into  a 
system  solution  that  includes  the  design,  manufacturing,  test  and  evaluation,  and  support  processes  and 
products.  The  systems  engineering  process  shall  establish  a  proper  balance  between  performance,  risk,  cost, 
and  schedule,  employing  an  iterative  process  of  requirements  analysis,  functional  analysis  and  allocation,  design 
synthesis,  verification,  and  system  analysis  and  control. 

Test  and  Evaluation  (T&E)  shall  ensure  that  all  end  products  are  tested  and  evaluated  to  the  full  requirements  in 
the  Verification  Plan  and  the  TEMP.  This  may  include  Ranges  (land  and  sea),  facilities  and  laboratories,  human 
factors,  aircraft/ship  and  related  systems. 

Tools 

Modeling  &  Simulation.  Electronic  &  Mechanical  Design  Analysis  using  Computer  Aided  Design  (CAD), 
Computer  Aided  Manufacturing  (CAM),  and  Electronic  Design  Automation  (EDA)  tools  shall  be  used  to  verify, 
validate,  and  analyze  the  design  for  compliance  against  the  requirements. 

Stress  Testing  at  above  normal  loads  shall  be  performed  on  the  system/subsystem/components  to  ensure  that  the 
system  can  handle  stress  above  the  system  operational  requirements.  These  above  normal  loads  are  increased  to 
determine  the  system’s  breaking  point;  these  tests  are  important  for  evaluating  the  robustness  of  the  system  and 
its  components. 

Software  Analysis  Tools  shall  be  used  to  perform  an  Independent  Verification  and  Validation  (IV&V)  of  the 
system  software  processes.  A  Systems  Assessment  shall  evaluate  the  requirements,  design,  testing,  and 
processes  of  the  system  design  and  shall  identify  risks  associated  with  mission  requirements  and  shall  make 
recommendations  for  corrective  action. 

Integrated  Architecture  products  shall  be  used  to  verify,  validate,  and  analyze  the  design  for  compliance  against 
the  requirements. 

Requirements  Management  Tools  Summary:  http://INCOSE.org/tools/tooltax.html 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 
ANSI/EIA  632  (Para.  4.5.2)  Processes  for  Engineering  a  System 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Verification  process  areas 

TE000-AB-GTP-010  Parts  Derating  Requirements  and  Applications  Manual  for  Navy  Electronic  Equipment 

Equivalent  to  MIL-STD-2164  Environmental  Stress  Screening  Process  for  Electronic  Equipment 

Equivalent  to  MIL- STD-454  Standard  General  Requirements  for  Electronic  Equipment 

DoD  4245.7-M  Transition  From  Development  to  Production 

NAVSO  P-6071  Best  Practices  -  The  Transition  from  Development  to  Production 

Metrics  and  Measures 

Test  Schedules  (including  dates,  milestones,  etc.)  are  met. 
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The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  show  that:  (1)  the  system  design  solution  appropriately  integrates  the  end 
products,  the  enabling  products,  and  the  external  interfacing  products  as  appropriate  to  the  level  of  the  system 
structure  and  enterprise-based  life-cycle  phase;  (2)  the  functional  and  performance  requirements  of  the  selected 
physical  solution  representation  are  satisfied;  (3)  the  functions  of  the  selected  physical  solution  representation 
have  been  implemented  correctly;  and  (4)  the  system  constraints  are  satisfied,  including  physical,  functional, 
and  human  interfaces. 


Sub-process  31  -  End  Product  Verification 

The  developer  shall  verify  that  an  end  product  (“as  built”  production  representative)  to  be 
delivered  to  an  acquirer  conforms  to  its  specified  requirements. 


End  Product  Verification  methods  include  any  and  all  of  the  following:  inspection,  analysis,  simulation, 
demonstration,  and  ground/flight  test  of  “as  built”  production  representative  system. 

Preceding  Process 
Supply  Process 

Sub-process  1 :  Product  Supply 
Planning  Process 

Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
System  Verification  Process 

Sub-process  30:  Design  Solution  Definition 

Inputs 

•  End  Products  (“as  built”  production  representative)  (SP  1) 

•  Enabling  products  (SP  1) 

•  Test  and  Evaluation  Master  Plan  (TEMP)  (SP  7) 

•  Verification  Plan  (SP  7),  including  the  Verification  Compliance  Requirement  Matrix  (VCRM) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Capability  Development  Document  (CDD)  or  Capability  Production  Document  (CPD)  -  (formerly 
Operational  Requirements  Document  (ORD)  (SP  14) 

•  Specified  requirements  (SP  19) 

•  Demonstration  Test  Readiness  Report  (DTRR)  (SP  30) 

•  Design  solution  verification  report  (SP  30) 

•  Design  solution  deficiency  and  discrepancy  report  (SP  30) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents  (approved  Test  Plan  including  risk 
mitigation). 

Tasks 

The  developer  (either  a  government  test  team  or  in  most  cases,  now,  an  Integrated  Test  team,  inclusive  of  the 
prime  contractor)  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider 
include  the  following: 

a)  Plan  the  end  product  (system  and  subsystem,  “as  built”)  verification  in  accordance  with  the 
Verification  Plan,  agreement  (normally  associated  with  detailed  developmental  test  plans), 
applicable  enterprise-based  life-cycle  phase,  and  level  in  the  system  structure.  This  shall  include: 
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•  selection  and  definition  of  the  appropriate  method  for  end  product  (system  /subsystem,  “as  built”) 
verification  (which  should  come  from  a  detailed  Test  Plan  that  describes  the  methods  and 
processes  to  be  used  in  verifying  compliance  against  the  specified  requirements)  since  methods 
will  be  based  on  platform,  or  item  under  test; 

•  verification  procedures  to  be  followed  for  the  method  selected  (including  the  purpose  and 
objective  of  each  procedure,  pretest  action,  and  post-test  action;  and  the  criteria  for  determining 
success  or  failure  of  the  procedure)  (NA  VAIR  Specific:  Ensure  these  can  be  performed  in 
accordance  with  NAVAIR  Instruction  3960.2,  Test  and  Evaluation.); 

•  establishment  and  checkout  (for  example,  adequacy  and  completeness)  of  the  environment  (for 
example,  climatic  conditions,  equipment,  facilities,  and  measuring  devices,  etc.)  in  which  the 
verification  method  and  procedures  will  be  implemented;  and 

•  assurance  that  the  test  articles  are  on  hand,  assembled,  or  integrated  with  the  verification 
environment  according  to  verification  plans  and  schedules,  and  that  appropriate  sets  of  specified 
requirements  are  available. 

Coordination  with  test  ranges,  and  other  testing  evaluation  and  engineering  facilities  is  a  must  to 
ensure  necessary  and  satisfactory  testing  support  will  be  provided  when  required.  Coordination  with 
the  prime  contractor,  facility  managers,  test  squadrons  and  platform  coordinators  is  absolutely  required 
to  ensure  test  articles  are  on  hand  and  prepared  for  test. 

For  flight  test,  FLIGHT  CLEARANCES  are  required,  and  the  NAVAIR  flight  clearance  process  will 
be  followed.  Depending  upon  the  system/subsystem  under  test  and  circumstances,  coordination  with 
test  squadron  Project  Liaison  Office  is  required  to  ensure  appropriate  clearance  type  is  obtained. 

b)  Verify  the  end  product  (system/subsystem, 66 as  built”),  using  the  selected  methods  and 
procedures  within  the  established  verification  environment  (regardless  of  methodology  selected,  a 
common  method  of  documentation  for  data  tracking  purposes  should  be  employed)  to: 

1)  collect  and  evaluate  verification  outcomes  to  either  show  compliance  or  identify  variances 
(unverifiable  requirements  and  constraints);  and 

2)  (for  variances  not  caused  by  poor  test  conduct  or  conditions)  complete  appropriate  tasks  of  the 
Planning  Process,  the  Control  Process,  the  Requirements  Definition  Process,  and  the  Solution 
Definition  Process  to  resolve  variances,  and  then  repeat  this  set  of  End  Product  Verification  tasks. 
(The  generation  of  deficiency  reports,  YELLOW  SHEETS,  White  Sheets,  and  System/Software 
Trouble  Reports,  etc.,  are  all  used  to  document  variances  in  the  systems/subsystems  under  test.) 

Data  collection  is  determined  by  the  type  of  tests  (flight  or  facility)  being  performed.  For  flight  and 
aircraft  ground  based  testing,  the  availability  of  onboard  data  collection  (instrumentation  requirements 
must  be  coordinated  with  test  and  evaluation  if  existing  onboard  equipment  is  not  capable  of  recording 
test  data  for  the  evaluation)  and  range/facility  capabilities  (real  time  telemetry,  playback,  etc).  All 
laboratory/facility  testing  must  be  coordinated  with  the  appropriate  lab  managers  to  ensure  adequate 
and  satisfactory  data  collection  is  available.  Data  is  centrally  collected  into  an  integrated  enterprise 
data  respository  for  requirements  verification.  Any  system  requirements  that  are  not  controllable  and 
observable  shall  be  reported  as  an  unverifiable  requirement  and  reported  to  Sub-process  16  via  Sub¬ 
process  25  but  should  be  confirmed  as  part  of  task  b)  1)  above  as  well. 

c)  Reverify  according  to  a  redesigned  verification  plan,  test  method,  or  procedure  when  variances 
are  determined  to  be  caused  by  poor  verification  or  inadequate  verification  environmental 
preparation.  Additional  testing,  as  well  as  regression  testing,  may  be  required  based  on  type  and 
magnitude  of  fixes.  (The  amount  of  regression  testing  required  is  platform,  and  system  under  test, 
driven.)  If  variances  are  caused  by  poor  verification,  return  to  first  bullet  of  task  a). 

d)  Record  verification  results,  including  corrective  actions  taken;  lessons  learned;  outcomes 
achieved;  Trade-off,  effectiveness,  and  risk  analyses  completed  with  resulting  key  decisions;  test 
activities  completed;  variances;  and  the  verified  end  products  in  the  established  enterprise  data 
respository.  A  test  report  (the  fomat  to  be  agreed  upon  by  both  the  testing  organization  and  the 
sponsoring  activity)  is  generated  to  provide  Program  Executive  Officers  and  Program  Managers  with 
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the  appropriate  level  of  engineering  information  to  make  educated  acquisitional  decisions  and  approve 
test  articles  for  final  operational  evaluation  or  intermediate  developmental  technical  evaluation. 

NA  VAIR  Specific :  Depending  upon  the  specific  aviation  program,  the  report  is  also  required  to  be 
provided  to  the  Naval  Technical  Aviation  Board  (NTAB)  per  NAVAIR  Instruction  3960.2. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Detailed  developmental  test  plans  (SP  31) 

•  Developmental  test  methods  (SP  31) 

•  Developmental  test  procedures  (SP  31) 

•  End  product  deficiency  and  discrepancy  reports 
(SP  10,  11,19) 

•  Developmental  Test  /  Operational  Test  (DT/OT)  Transition  Report  (SP  33) 

•  Report  of  Test  Results  with  limitations  and  constraints  for  Operational  Test  (OT)  (SP  33) 

•  Operational  Advisory  Document  (SP  33) 


Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agent.  (Completion  of  the  Verification  phase 
evaluated  results  and  reported  conclusions.) 

Next  Processes 
Assessment  Process 

Sub-process  10:  Progress  Against  Requirements 
Sub-process  11:  Technical  Reviews 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
End  Products  Validation  Process 

Sub-process  33:  End  Products  Validation 

Agents 

T&E 

R&M 

Systems  Engineering 
Human  Factors 
Acquirer 
PEO/PM 

Operators  /  Users  (for  NAVAIR  this  includes  AIR-5.5,  OPTEVFOR,  Fleet) 

Developer  /  Contractor  (Various) 

Tools 

Ranges  (for  NAVAIR  this  is  primarily  AIR  5.1  /  AIR  5.2  with  selected  other  outside  organizations  for  flight 
test) 

Test  Plans  (system,  subsystem,  and  integrated) 

Facilities/Labs  (for  NAVAIR  this  is  primarily  5.x  and  4.x  for  ground  tests;  could  be  developer/contractor) 
Aircraft  and  systems  under  test,  and  ALL  supporting  systems  under  test 
Flight  Clearance 
Deficiency  Database 

Integrated  architecture  products  shall  be  used  to  verify  that  the  integrated  end  products  meet  operational 
requirements  and  to  ensure  that  systems/sub-systems,  data/information,  materiel  and  services  can  operate 
effectively  together. 
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References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Verification  process  areas 
NAY  AIR  Test  and  Evaluation  Instruction  3960.2  series 
NAVAIRNTAB  Instruction  3960.5 

NATOPS  Flight  and  Weapon  Systems  Manual  (for  each  platform) 

Range  Safety  Operation  Guides  (for  each  range  operated  on) 

Test  Squadron  Standard  Operating  Procedures  (SOPs) 

Facility  SOPs 

U.S.  Naval  Test  Pilot  School  Flight  Test  Manual 
Software  Requirements  Specifications 
Manufacturer’s  specifications 
SARs  /  STRs 

Metrics  and  Measures 

Deficiencies  (Part  I,  II,  III),  number  and  severity 

•  Specification  Compliance,  yes/no  and  why 

•  TEMP  Compliance,  yes/no  and  why 

•  Mission  Relation/Impact,  descriptive 

Earned  Value  Management  (cost,  performance,  test  completion,  ground/lab/flight  hours,  and  data  points) 

Test  Schedule  (including  deliverable  dates,  milestones,  and  LRIP),  performance  relative  to  End  Product 
Deficiency  Reports  (Software  and  Hardware) 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  show  that  the  integrated  composite  of  end  products:  (1)  complies  to  its 
specified  requirements;  (2)  functions  together  with  other  system  end  products  and  with  interfacing  products 
throughout  the  performance  envelope;  and  (3)  is  ready  for  delivery  to  the  acquirer,  in  accordance  with  the 
agreement. 


Sub-process  32  -  Enabling  Products  Readiness 

The  developer  shall  determine  readiness  of  enabling  products  for  development,  production, 
test,  deployment/installation,  training,  support/maintenance,  and  retirement  or  disposal. 


This  sub-process  determines  the  readiness  of  enabling  products  furnished  by  the  developer  to  support  each  life¬ 
cycle  phase  of  the  product. 

Preceding  Processes 
Supply  Process 

Sub-process  1 :  Product  Supply 
Planning  Process 

Sub-process  5:  Technical  Effort  Definition 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 


Inputs 
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•  Enabling  Products  (SP  1) 

•  List  of:  Methods  and  Tools,  Facilities,  Equipment,  Training  (SP  5) 

•  Specified  requirements  (SP  19) 

•  Enabling  products  development  projects  (SP  19) 

Categories  and  examples  of  enabling  products: 

-  Fleet  Assets  -  fleet-owned  assets  being  modified  (ex.  Mission  computer,  radar  system,  flight  control 
system),  operational  assets  (support  aircraft,  ship  assets,  drones,  weapon  targets,  satellites),  etc. 

-  Development  -  CAE  Tools,  Prototypes,  Life  cycle  analysis,  Laboratories/Facilities,  Requirements 
Management  &  System  Architecture  Database,  Software  Development  Facility,  etc. 

-  Production  -  Tooling  and  Facilities,  Manpower,  etc. 

-  Test  -  Test  Equipment  &  Software,  Verification  Plans  &  Procedures,  Test  Ranges,  GFE,  etc. 

-  Deployment  -  Staging  Facilities,  Warehouses,  Shipping  Containers,  etc. 

-  Training  -  Class  Rooms,  Flight  Simulator,  Instructors,  etc. 

-  Support  -  Repair  Facilities,  Diagnostic  Equipment,  Shipping  Services,  Staffing,  etc. 

-  Disposal  -  Disposal  site,  Refurbishment  Facilities,  Removal  Tools,  Safety  Bulletins,  etc. 

Further  description  of  enabling  products  can  be  found  in  Section  6. 1.1. 4  and  Appendix  F,  specifically 
Figure  F.3. 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include 
the  following: 

a)  Plan  enabling  product  readiness  determination  and  associated  process  proofing  in  accordance 
with  the  appropriate  plan,  maturity  of  related  end  products,  agreement,  applicable  enterprise-based  life- 
cycle  phase,  and  level  in  the  system  structure.  Include: 

•  selection  and  definition  of  the  appropriate  method  for  the  enabling  product  readiness 
determination  and  for  proofing  for  each  applicable  associated  process; 

•  readiness  determination  procedures  to  be  followed  for  the  method  selected,  the  purpose  and 
objective  of  each  procedure,  pre-test  and  post-test  actions,  and  the  criteria  for  determining  the 
success  or  failure  of  the  procedure; 

•  establishment  and  checkout  (for  example,  adequacy  and  completeness)  of  the  environment  (for 
example,  climatic  conditions,  equipment,  facilities,  and  measuring  devices,  etc.)  in  which  the 
readiness  determination  method  and  procedures  will  be  implemented;  and 

•  assurance  that  required  information  regarding  the  status  and  maturity  of  enabling  product 
development  or  requirements  definition  is  available,  and  that  non-developmental  enabling 
products  are  available  and,  if  appropriate,  integrated  with  the  environment  according  to 
appropriate  plans  and  schedules. 

A  comprehensive  plan  to  conduct  the  readiness  review  should  be  developed  and  agreed-to  by  the 
contractor  and  government.  Plan  should  include  resources  needed  to  conduct  review,  method  of 
establishing  contractor’s  readiness,  environment  or  facilities  necessary  for  the  assessment,  metrics  to 
ensure  mitigation  of  supplier’s  risk,  and  follow-up/corrective  action  plans. 

b)  Do  the  planned  enabling  product  readiness  determination  and  associated  process  proofing,  using 
the  selected  methods  and  procedures  within  the  established  environment  to: 

1)  collect  and  evaluate  readiness  determination  outcomes  to  either  show  compliance  or  identify 
variances  (untraceable  requirements  and  constraints,  anomalies,  variations,  voids,  and  conflicts); 
and 

2)  (for  variances  not  caused  by  poor  readiness  determination,  or  process  proofing  conduct  or 
conditions)  complete  appropriate  tasks  of  the  Planning  Process,  Control  Process,  Requirements 
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Definition  Process,  and  Solution  Definition  Process  to  resolve  variances,  and  then  repeat  the 
readiness  determination  or  proofing. 

Readiness  Reviews  should  be  conducted  to  assess  risk  of  enabling  products  supporting  each  life-cycle 
phase  of  the  product.  Actions  (with  milestones)  to  mitigate  risk  should  be  identified  in  readiness 
reports  to  stabilize  product  configuration  and  minimize  change  activity  in  later  phases.  Examples  of 
Readiness  Review  Reports  include  the  Integrated  Training  Plan,  Production  Readiness  Review  Report, 
Initial  Operating  Supportability  Capability  Review  Report,  and  Logistics  Support  Analysis.  Any 
design,  test,  manufacturing,  logistics,  and  disposal  issue  should  be  identified  in  the  readiness  reviews 
for  an  effective  product  development. 

c)  Reaccomplish  readiness  determination  according  to  redesigned  plans,  test  method,  or  procedure 
when  variances  were  determined  to  be  caused  by  poor  readiness  or  proofing  conduct,  or  by 
inadequate  environmental  preparation.  A  follow-up  or  another  readiness  review  can  be  conducted 
if  the  risk  was  considered  excessive  in  the  orginal  readiness  review. 

Supplier  must  provide  evidence  that  risk  has  been  effectively  mitigated  to  ensure  a  smooth  transition 
into  the  next  planned  life-cycle  phase.  After  exit  criteria  has  been  met  and  risk  has  been  lowered,  the 
supplier  is  ready  to  enter  the  next  planned  life-cycle  phase. 

d)  Record  readiness  determination  and  process  proofing  results,  including  corrective  actions  taken; 
lessons  learned;  outcomes  achieved;  trade-off,  effectiveness,  and  risk  analyses  completed,  with 
resulting  key  decisions;  test  activities  completed;  variances;  and  the  verified  enabling  products  and 
proofing  of  associated  processes  in  the  established  enterprise  data  repository. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Enabling  Products  Readiness  Determination  (SP  12,  21) 

•  Enabling  Products  Readiness  Assessment  Plan  (SP  32) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
Control  Process 

Sub-process  12:  Outcomes  Management 
Transition  to  Use  Process 

Sub-process  21:  Transition  to  Use 
System  Verification  Process 

Sub-process  32:  Enabling  Products  Readiness 

Agents 

System  Engineering 

Logistics 

T&E 

Training 

Manufacturing 

Program  Manager  (PM) 

All  of  these  agents  for  both  contractor  and  government  are  involved  in  ensuring  the  readiness  of  enabling 
products. 

Tools 

Enterprise  Data  Respository 
Integrated  Architecture  Products 
Manufacturing  Tooling 
TPM  Tracking  tools/Schedules 
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Test  Equipment  &  Software 

Statistical  Process  Control 

Manufacturing  Simulations 

CAD/CAM 

Removal  Tools 

Flight  Simulators 

Training  Manuals 

Readiness  Archives  and  Databases 

These  are  examples  of  tools  used  to  ensure  readiness  of  enabling  products.  This  list  of  tools  can  become 
exhaustive  depending  on  the  enabling  product. 

References 

Standard  across  ah  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Product  Integration  and  Verification  process 
areas 

DOD  5000.2R,  Parts  3.3,  5.2,  &  7.4 
MIL-STD-1521B 
MIL-STD-499B,  Parts  5.5  &  5.7 

NAVSO  P-6071  Best  Practices,  Sections  5.0,  6.0,  7.0,  9.0,  &  10.4 

DOD  4245.7-M  Transition  from  Development  to  Production,  Sections  4.0, 5.0,  6.0,  8.0,  &  9.0 

DAU  Program  Manager’s  Tool  Kit 

Metrics  and  Measures 

Adherence  to  Schedule  and  Progress  versus  Plan 
Sub-Process  Execution  Time  and  Cost 
System  Definition  Detail 

Technical  Performance  Measurement  Resolution  (Availability,  Reliability,  Capability,  Effectiveness,  etc.) 
Process  Control  Matrices 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  show  that:  (1)  associated  process  requirements  for  production,  test, 
deployment,  training,  support,  and  disposal  have  been  identified;  (2)  plans  and  selected  methods,  procedures, 
and  tools  for  each  associated  process  will  be  able  to  accomplish  their  intended  purpose;  (3)  enabling  product 
development  for  each  associated  process  will  be  completed  and  enabling  products  will  be  available  to  provide 
the  required  support  functions  to  the  intended  end  product;  and  (4)  associated  processes  are  properly  proofed 
(for  example,  proof  test  of  the  manufacturing  process  for  rate  production)  against  requirements  and  can  perform 
their  purpose  with  respect  to  support  of  the  intended  end  product. 


NOTE  -  For  each  associated  process,  enabling  products  requiring  development  will  go  through  both  design 
solution  verification  and  end  product  verification  as  the  processes  of  this  Guide  are  implemented  for  that 
development.  Off-the-shelf  or  reused  enabling  products  will  be  validated  against  the  acquirer  requirements, 
when  appropriate.  These  non-developmental  enabling  products  will  be  required  for  verification  of  physical 
and  functional  interfaces  with  their  related  end  products  during  the  associated  end  product  verification. 
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4.5.4  End  Products  Validation  Process 


The  End  Products  Validation  Process  is  used  to  demonstrate  that  the  products  to  be  delivered,  or  that  has  been 
delivered,  satisfy  the  validated  acquirer  requirements  (for  example,  customer,  user,  or  operator  requirements,  or 
assigned  requirements)  that  were  input  to  the  system  design  processes  and  that  are  applicable  to  the  resulting 
end  products. 

This  process  is  usually  interpreted  to  mean  the  Operational  Test  of  the  system.  Not  all  systems  are  subject  to 
formal  Operational  Test  (OT),  and  this  process  may  have  to  be  tailored  for  these  systems.  Also,  when  speaking 
of  software  development,  the  term  “validation”  takes  on  a  different  meaning  and  is  defined  in  IEEE/EIA- 12207. 
When  a  software  development  becomes  a  major  system  and  subject  to  OT,  this  process  applies  over  and  above 
the  software  definition  of  “validation”.  Operational  Testing  is  usually  conducted  in  phases  as  part  of  the  life- 
cycle  development  of  the  system.  Early  testing  is  usually  conducted  as  an  Operational  Assessment  (OA)  and 
then  proceeds  through  the  various  OT  test  periods  to  OPEVAL  and  FOT&E. 


Sub-process  33  -  End  Products  Validation 

The  developer  shall  ensure  that  an  end  product,  or  an  aggregation  of  end  products,  conforms 
to  its  validated  acquirer  requirements. 


Preceding  Processes 
Supply  Process 

Sub-process  1 :  Product  Supply 
Planning  Process 

Sub-process  7:  Technical  Plans 
Requirements  Definition  Process 

Sub-process  14:  Acquirer  Requirements 
System  Verification  Process 

Sub-process  31:  End  Product  Verification 

Inputs 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Validation  Plan  (Operational  Test  Plan)  (SP  7) 

•  Test  and  Evaluation  Management  Plan  (TEMP)  (SP  7) 

•  Validation  Plan  (known  here  as  the  Operational  Test  Readiness  Review  (OTRR)  Plan)  (Internal  or  SP  7) 

•  Initial  Capabilities  Document  (ICD)  -  formerly  Mission  Needs  Statement  (MNS)  (SP  14) 

•  Capability  Development  Document  (CDD)  -  formerly  Operational  Requirements  Document  (ORD)  (SP  14) 

•  Developmental  Test/Operational  Test  (DT/OT)  Transition  Report  (SP  31) 

•  Report  of  Test  Results  with  limitations  and  constraints  for  Operational  Test  (OT)  (SP  31) 

•  Operational  Advisory  Document  (SP  31) 

Entry  Criteria 

Inputs  have  been  reviewed  and  approved  by  the  appropriate  agent.  For  most  programs,  the  appropriate 
Development  Test  (DT)  must  have  been  successfully  completed  and  a  DT  report  issued. 

Tasks 

The  developer  should  plan  and  do  the  appropriate  tasks  to  complete  this  sub-process.  Tasks  to  consider  include 
the  following: 

a)  Determine  the  type  of  end  product  validation  required  and  the  exit  criteria,  including  the 

acquirer  requirements  applicable  to  the  sytem  end  products  being  validated.  This  task  is  usually 
encompassed  in  the  Operational  Test  Readiness  Review  (OTRR).  A  succesful  OTRR  will  result  in  a 
certification  message  to  Commander,  Operational  Test  and  Evaluation  Force  (COMOPTEVFOR) 
stating  that  the  system  is  ready  for  operational  testing.  This  is  achieved  after  review  of  the  CDD, 
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TEMP,  DT  Test  Report,  OT  Test  Plan  and  other  inputs  to  confirm  that  the  system  will  be  tested  in  an 
appropriate  manner  against  the  correct  criteria.  As  necessary,  waivers  to  requirements  and  limitations 
to  testing  must  be  defined  during  this  task. 


NOTES: 

1  For  a  system  that  is  an  aggregation  of  end  products  (see  building  block  discussion  in  Subsection  6.1), 
the  individual  end  products  and  the  aggregation  of  end  products  are  to  be  validated. 

2  The  types  of  end  product  validation  include:  (1)  validation  against  validated  acquirer  requirements  in 
the  anticipated  usage  environment  with  test  conditions  that  span  the  expected  range  of  actual  operating 
conditions;  (2)  certification  tests  against  established  certification  requirements;  (3)  acceptance  tests,  using 
operational  processes  and  personnel  in  an  operational  environment;  or  (4)  as  specified  in  the  agreement. 


b)  Acquire  the  test  article,  or  aggregation  of  end  products,  for  the  validation  as  appropriate  to  the 
enterprise-based  life-cycle  phase  and  level  of  system  structure.  Test  articles  for  OT  must  be 
representative  of  production  and  are  usually  procured  as  part  of  a  Low  Rate  Initial  Production  (LRIP) 
contract.  In  early  phases  where  an  OT  is  being  conducted,  the  test  article  may  be  a  prototype  or  even  a 
model  as  described  below,  but  any  model  must  be  certified  by  the  COMOPTEVFOR.  The  number  of 
test  articles  and  their  configuration  need  to  be  planned  in  conjunction  with  the  Test  and  Evaluation 
Management  Plan  (TEMP).  The  “test  article”  should  include  any  support  equipment,  trainers,  or  other 
items  necessary  to  test  the  article  under  operationally-representative  conditions. 


NOTES  -  The  test  article  is  typically  the  product,  or  an  aggregation  of  products,  that  is  to  be 
delivered  or  that  has  been  delivered  and  that  has  already  been  verified.  In  early  enterprise-based 
life  cycle  developments,  the  product  or  aggregation  of  products  undergoing  validation  can  be  a 
virtual  prototype,  breadboard,  brassboard,  or  model.  Thus,  a  detailed  simulation,  operated  so  that 
acquirer  perceptions  can  be  evaluated,  is  a  possible  means  of  validation. 


c)  Conduct  the  end  products  validation  in  accordance  with  the  Validation  Plan,  as  required  in  the 
agreement,  to  show  conformance  with  appropriate  requirements;  collect  and  analyze  validation 
outcomes  to  identify  any  variances;  and  do  appropriate  process  tasks  to  resolve  variances  and 
repeat  appropriate  verifications  and  validations.  Actual  conduct  of  the  test  is  the  responsibility  of 
COMOPTEVFOR.  A  final  report  will  document  the  validation  results.  Even  a  successful  OT  will 
often  list  deficiences  that  need  to  be  corrected  at  a  later  time  or  phase. 

d)  Revalidate  with  improved  or  corrected  procedures  and  equipment  when  variances  were  caused 
by  poor  test  conduct  and  conditions.  This  task  is  applicable  when  the  Operational  Test  Activity  has 
scored  the  system  being  validated  as  “not  operationally  suitable  or  effective,”  and  the  operational  test 
appears  to  be  flawed  for  the  reasons  stated.  If  the  Operational  Test  Activity  concurs,  then  task  c  ) 
above  should  be  repeated  using  correct  procedures  or  equipment. 

e)  Record  the  validation  outcomes,  procedures,  assumptions,  lessons  learned,  and  other  pertinent 
information  about  the  validation  and  results  in  the  established  enterprise  data  repository,  to 
provide  traceability.  Leads  to  Sub-process  12. 

Outputs  (List  of  sub-processes  where  output  is  used  may  include  the  originating  sub-process.) 

All  outputs  should  be  archived  (SP  12) 

•  Operational  Test  Readiness  Review  (OTRR)  Plan  (SP  33) 

•  Operational  Test  Readiness  Review  (OTRR)  certification  message  (SP  2) 

•  Operational  Test  /  Follow-On  Test  &  Evaluation  (OT/FOT&E)  Report  (SP  19,  20) 

Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the  appropriate  agents. 

Next  Processes 
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Acquisition  Process 

Sub-process  2:  Product  Acquisition 
Control  Process 

Sub-process  12:  Outcomes  Management 
Solution  Definition  Process 

Sub-process  19:  Specified  Requirements 
Implementation  Process 

Sub-process  20:  Implementation 

Agents 

OPTEVFOR,  DOT&E,  Systems  Engineering,  T&E,  COMOPTEVFOR 
Tools 

Modeling  &  Simulation  (M  &  S),  Hardware  In-the-Loop  (HIL),  Software  In-the-Loop  (SIL),  Flight  Test 
References 

Standard  across  all  systems  engineering  efforts: 

•  DoD  5000  Series 

•  AT&L  Knowledge  Sharing  System  (AKSS) 

•  FAR/DFARs 

•  C4ISR  Architecture  Framework 

•  Joint  Technical  Architecture 

•  Defense  Acquisition  University:  Systems  Engineering  Fundamentals 

•  INCOSE  Systems  Engineering  Handbook 

Capability  Maturity  Model®  Integration  (CMMI— ),  2001:  Validation  process  areas 
DRAFT  MIL-STD-499B  Systems  Engineering 
NAVAIRINST  3960.2 C  Test  and  Evaluation 
IEEE/El  A- 12207 

Metrics  and  Measures 

OTRR  is  achieved  within  program  schedule. 

Operational  test  procedures  and  processes  are  carried  out  according  to  the  TEMP. 

The  expected  outcomes  for  these  representative  tasks  are  provided  in  Appendix  C.  The  outcomes  associated 
with  completing  this  sub-process  provide  the  end  products  that  conform  with  acquirer  requirements  stated  in  an 
agreement,  including  any  approved  changes,  or  certification  or  acceptance  criteria,  as  appropriate. 


NOTES 

1  The  key  difference  between  end  product  validation  and  end  product  verification  is  that  end  product 
validation  answers  the  question:  Does  the  delivered  end  product  conform  to  the  validated  input  acquirer 
requirements,  certification  criteria,  or  acceptance  criteria,  as  applicable?  End  product  verification  answers 
the  question:  Does  the  output  end  product  comply  to  the  output  specified  requirements  from  which  the  end 
product  was  built,  coded,  procured,  or  assembled  and  integrated? 

2  Processes  or  manual  procedures  that  are  part  of  the  defined  solution  are  implicitly  included  in  this 
validation,  since  they  are  a  type  of  product. 

3  Sub-process  33  addresses  the  validation  of  each  end  product,  or  aggregation  of  end  products,  against 
validated  acquirer  requirements.  There  can  be  cases  where  it  is  also  appropriate  to  validate  against  other 
stakeholder  requirements. 

4  In  addition,  there  can  be  cases  where  it  is  appropriate  to  validate  against  actual  needs  and  expectations  of 
end  users  in  their  environment  under  real-world  conditions.  This  is  called  by  various  names:  market  trial, 
beta  testing,  or  operational  test  and  evaluation. 
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5  Application  context 


This  section  describes  the  application  context  for  the  sub-processes  of  this  Guide.  Figure  5  shows  external 
enterprise  and  project  factors  that  have  the  potential  to  affect,  or  be  affected  by,  project  interfaces 


External  Environment 


•  LAWS  &  REGULATIONS  •  LEGAL  LIABILITIES  •  SOCIAL  RESPONSIBILITIES  •  TECHNOLOGY  BASE 

•  LABOR  POOL  •  COMPETING  PRODUCTS  •STANDARDS  &  SPECIFICATIONS  •  PUBLIC  CULTURE 

_  Enterprise  Environment  _ 


•  POLICIES  &  PROCEDURES  •  STANDARDS  &  SPECIFICATIONS 

•  GUIDELINES  •  DOMAIN  TECHNOLOGIES  •  LOCAL  CULTURE 


Project  Environment 


•  DIRECTIVES  &  PROCEDURES  •  PLANS  •  TOOLS  •  PROJECT  REVIEWS  •  METRICS 


Process  Groups  for 

Project  Support 

/ 

Engineering  Systems  \ 

Project  Management 
Agreement  Support 


Project  A 


•  Acquisition  &  Supply 

•  Technical  Management 

•  System  Design 

•  Product  Realization 

•  Technical  Evaluation 


Project  B 


Project  C 


Enterprise  Support 


•  Investment  Decisions 

•  External  Agreements 

•  Infrastructure  Support 

•  Resource  Management 

•  Process  Management 

•  Production 


Field  Support 


Figure  5  -  Context  for  application  of  this  Guide 

5.1  Enterprise  factors 

The  enterprise  is  the  context  in  which  the  process  requirements  of  this  Guide  are  intended  to  be  adopted, 
directed,  and  implemented.  The  enterprise  is  the  source  of  project  start-ups  and  of  project  cancellations,  and  is 
the  source  of  infrastructure  and  resource  support.  Enterprises  respond  to,  as  well  as  create,  the  markets  for  the 
system  products  created  by  projects  within  the  enterprise.  The  enterprise  further  manages  the  multiple  projects 
within  the  enterprise  to  most  effectively  apply  resources  and  use  the  infrastructure.  The  enterprise  also 
establishes  constraints  of  technologies  used  in  existing  product  lines,  as  well  as  manufacturing  and  test 
facilities,  and  support  service  limitations  that  constrain  project  performance. 

It  is  in  this  context  that  the  enterprise  prepares  policies  and  procedures  to  create  or  cancel  projects,  and  by 
which  projects  perform  the  processes  of  this  Guide. 

5.2  Project  factors 
5.2.1  Enterprise  support 

Projects  create  systems  consistent  with  the  business  strategy  of  the  enterprise  and  within  the  constraint  of  the 
enterprise  factors  cited  in  Subsection  5.1.  Specifically,  the  following  support  is  to  be  expected  from  the 
enterprise: 
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a)  Investment  decision  support,  including  business  needs  assessments,  selection  of  new  start  projects, 
determination  of  project  continuance,  and  allocation  of  financial  resources  for  equipment,  tools,  and 
training; 

b)  Agreement  support,  including  contracting,  bid  and  proposal  funding,  proposal  preparation,  and  oversight 
(when  an  external  agreement  is  required); 

c)  Infrastructure  support,  including  research  and  development,  marketing,  facilities,  in-service  support, 
computer  services,  and  other  services  that  enable  the  project  to  meet  its  obligations; 

d)  Resource  management  support,  including  financial  management,  personnel  management,  training  and 
education  of  project  personnel,  office  and  computer  equipment,  maintenance,  and  shipping; 

e)  Process  management  support,  including  establishment  of  standard  procurement  processes  and  methods, 
guidelines  for  tailoring  adopted  processes  from  this  Guide,  selection  and  acquisition  of  tools,  assessment  of 
directed  process  implementation  and  monitoring  of  process  effectiveness,  and  improvement  of  processes; 

f)  Production  support,  including  fabrication,  construction,  manufacturing  capacity,  and  staffing;  equipment 
and  tools;  and  accomplishing  fabrication,  construction,  manufacturing,  quality  control,  and  testing; 

g)  In-service  support,  including  installation,  customer  support,  product  upgrades,  warranty  service,  field 
modifications,  on-site  consulting,  and  product  certification. 

The  availability  and  adequacy  of  enterprise  support  functions  determine  the  viability  of  a  project,  schedule  of 
project  tasks,  capability  to  satisfy  an  established  agreement  with  another  enterprise,  and  the  availability  of 
personnel  who  have  the  skills  and  knowledge  to  complete  project  responsibilities. 

5.2.2  Project  support  of  the  technical  process 

Projects  provide  the  context  in  which  a  system  is  engineered.  Projects  use  the  processes  from  this  Guide  as 
directed  by  enterprise  policies,  or  as  directly  adopted  by  the  project,  to  satisfy  agreements.  Directives  and 
procedures  are  prepared  by  the  project  to  guide  both  the  project  management  functions  and  the  technical  efforts 
applicable  to  the  specific  project.  In  this  context,  the  technical  efforts  to  meet  the  requirements  of  this  Guide 
require  project  functional  support.  Such  support  includes: 

a)  Agreement  support  including  preparing  appropriate  tasking  agreements  between  projects,  or  within  the 
project,  to  implement  the  planned  technical  effort,  and  providing  proposal  preparation  support,  as 
applicable. 

b)  Project  management  including  project  integration,  scope  management,  time  management,  cost 
management,  quality  management,  human  resource  management,  communication  management,  risk 
management,  and  procurement  management. 


NOTE  -  More  information  on  the  types  of  support  to  be  exptected  is  in  A  Guide  to  the  Project  Management 
Body  of  Knowledge  (PMBOK)  published  by  the  Project  Management  Institute. 


The  availability  and  adequacy  of  these  project  functions,  and  the  project  directives  and  procedures,  determine 
the  tasks  and  scope  of  the  processes  for  engineering  a  system.  The  enterprise  determines  the  tools,  equipment, 
and  metrics  to  be  used,  and  the  reporting  and  management  review  requirements. 

5.3  External  factors 

The  external  environmental  factors  that  can  affect  the  processes  for  engineering  a  system  include  local,  state, 
national,  and  international  laws  and  regulations;  potential  legal  liabilities;  social  responsibilities;  available 
technologies;  the  labor  pool;  competing  products  and  technologies;  and  national  or  international  standards  and 
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specifications.  Also,  the  processes  for  engineering  a  system  can  be  affected  by  external  agreements  for  upper  or 
lower  development  projects  and  requisitioned  end  products  and  be  existing  external  infrastructures  and  the 
physical  world. 

Systems  and  their  products  operate  with  organizations  and  personnel  who  use  the  end  products,  and  with  other 
operational  entities  that  provide  input  to  the  system,  or  otherwise  interact  with  the  system  products,  but  are  not 
part  of  the  system  under  development  and  are  not  controlled  by  the  developer.  The  interaction  and  interfaces 
(physical  or  functional)  between  the  system  products  and  their  external  operational  environment  can  affect  the 
inplementation  of  the  processes  used  for  engineering  the  project  system.  Changes  in  the  operational 
environment  can  strongly  affect  system  effectiveness  and  functionality.  System  performance  and  adequacy  also 
can  be  affected  by  the  system’s  ability  to  respond  both  to  the  operational  environment  and  to  changes  in  the 
environment. 

5.4  Influence  of  other  enterprise  projects 

Enterprises  often  have  more  than  one  development  project  at  once.  Two  such  projects  can  sometimes  benefit 
from  the  exchange  of  products,  for  example,  parts,  subassemblies,  or  data.  Agreements  between  such  projects 
are  established,  as  appropriate. 


6  Application  key  concepts 


This  section  describes  key  concepts  for  application  of  the  processes  of  Section  4  to  the  engineering  or 
reengineering  of  a  system.  There  are  two  aspects  to  this  section:  first,  the  system  to  which  the  processes  are 
applied;  and  second,  the  top-down  development  of  system  products  and  the  bottom-up  implementation  and 
realization  of  system  products.  The  first  is  the  basis  for  the  system  structure;  the  second  is  the  basis  of  an 
engineering  life  cycle. 

6.1  System  concept 

The  system  to  which  the  processes  of  Section  4  are  applied  consists  of  both  the  end  products  to  be  used  by  an 
acquirer  for  an  intended  purpose  and  the  set  of  enabling  products  that  enable  the  creation,  realization,  and  use  of 
an  end  product,  to  an  aggregation  of  end  products.  Enabling  products  are  used  to  perform  the  associated 
process  functions  of  the  system  -  develop,  produce,  test,  deploy,  and  support  the  end  products;  train  the 
operators  and  maintenance  staff  of  the  end  products;  and  retire  or  dispose  of  end  products  that  are  no  longer 
viable  for  use.  Both  the  end  products  and  the  enabling  products  are  either  developed  or  reused,  as  appropriate. 
The  relationship  of  these  system  elements  is  shown  in  Figure  6.1. 


Figure  6.1  -  System  concept 
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NOTE  -  The  above  system  concept  implicitly  includes  the  personnel  who  develop,  produce,  test, 
operate,  support,  and  retire  the  system  products,  as  well  as  both  those  who  train  others  involved  with 
these  system  functions,  and  the  human  factors  issues  and  concerns  associated  with  these  personnel. 

Such  personnel  and  human  factors  issues  are  included  in  the  application  of  the  processes  of  this  Guide  to 
the  building  block  structure  derived  from  this  system  concept. 

6.1.1  Building  block 

Ths  system  forms  the  basis  for  a  larger  structure,  called  the  building  block,  shown  in  Figure  6.1.1.  The  building 
block  provides  the  framework  for  application  of  the  processes  of  Section  4. 


Figure  6.1.1  -  Building  Block 

A  building  block  is  made  up  of  the  system  (gray  element),  one  or  more  end  products  (black  elements),  two  or 
more  subsystems  (gray  elements)  for  each  end  product,  and  the  ensemble  of  enabling  products  (white  elements). 
Each  end  product  and  each  enabling  product  includes  one  or  more  of  the  following;  hardware,  software, 
firmware,  personnel,  facilities,  data,  materials,  services,  and  processes.  The  following  information  can  be 
associated  with  each  element  within  the  building  block: 


a)  configuration  identification; 

b)  the  costs  to  be  collected; 

c)  identification  of  interfacing  elements  inside  and  outside  the  building  block; 

d)  specifications  relevant  to  the  element; 

e)  definition  of  work  to  be  done; 

f)  other  relevant  agreement  information. 


6.1. 1.1  System  element 


The  system  element  of  the  building  block  is  the  object  for  which  the  developer  defines  the  acquirer  and  other 
stakeholder  requirements  using  the  Requirements  Definition  Process. 
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6.1. 1.2 


End  product  element 


The  end  products  perform  the  operational  functions  for  the  system.  These  products  are  developed  using  the 
Solution  Definition  Process,  are  verified  against  the  specified  requirements  using  the  System  Verification 
Processs,  and  are  validated  against  acquirer  requirements  using  the  End  Products  Validation  Process. 

An  end  product  can  be  either  a  legacy  product  that  is  being  reengineered  or  a  product  that  the  enterprise  both 
has  the  expertise  to  make  and  has  similar  products  already  in  the  market  place.  Such  developments  are 
identified  as  precedented,  derivative,  or  next-generation.  When  the  specified  end  product  is  not  a  priori  known, 
or  when  the  enterprise  has  limited  experience  in  development  of  a  new  system,  the  development  is  identified  as 
precedented  or  as  a  new  concept. 


NOTE  -  Suppose  it  is  already  known  that  a  radio  set,  a  radar,  an  automobile,  or  another  specific  product 
(including  acquirer- furnished  equipment)  is  to  be  used  as  an  end  product.  Even  though  the  product  type 
is  known  (precedented),  the  specific  solution  for  this  next-generation  product  can  be  defined  using  the 
processes  of  this  Guide  to  satisfy  acquirer  requirements. _ 


An  end  product  can  be  self-contained  in  terms  of  its  use  and  operations.  It  also  can  be  an  item  that  has  no  use 
outside  a  larger  end  product,  but  that  is  developed  as  an  end  product  of  a  subsystem  (lower-layer  system 
building  block)  using  the  System  Design  Processes. 


NOTES 

1  Examples  of  self-contained  end  products  are  an  aircraft,  an  automobile,  a  communications  satellite, 
a  nuclear  reactor,  a  telecommunication  switching  module,  or  a  space  vehicle  that  is  delivered  to  an 
operator. 

2  An  end  product  could  also  be  any  of  many  products  that  make  up  a  self-contained  end  product. 
Examples  of  such  end  products  are  an  engine  or  a  radio  on  an  aircraft,  a  power  train  or  a  brake  for  an 
automobile,  a  solar  panel,  or  a  transmitter  for  a  satellite,  a  control  panel  or  a  control  valve  for  a  nuclear 
reactor,  a  switch  or  a  transducer  for  a  telecommunication  switching  module,  or  a  life  support  package  or 
a  hatch  door  for  a  space  vehicle.  Such  end  products  can  be  found  at  the  assembly,  subassembly,  line 
replaceable  unit,  component,  or  part  levels  of  a  system. 

3  The  end  product  element  is  black  to  represent  those  elements  of  the  building  block  that  are 
physically  integrated  with  end  products  of  upper-  and  lower-layer  building  blocks  to  form  a  composite 
end  product  and  eventually  a  self-contained  end  product. 

4  There  can  be  more  than  one  end  product  in  a  building  block.  In  such  cases,  the  system  consists  of  an 
aggregation  of  end  products,  plus  their  enabling  products. 


6.1. 1.3  Subsystem  elements 

If  end  products  cannot  be  manufactured  or  are  not  off-the-shelf  products  that  can  be  reused  and  purchased  from 
another  supplier,  subsystems  of  an  end  product  are  developed  using  the  processes  of  this  Guide.  Each  end 
product  that  is  developed  consists  of  two  or  more  subsystems  (gray  elements).  When  a  subsystem  is  developed, 
another  lower-layer  building  block  is  established  (see  Subsection  6.2).  The  hierarchy  of  such  building  blocks  is 
called  the  system  structure. 

6.1 .1 .4  Enabling  product  elements 

Enabling  products  perform  the  associated  process  or  non-operational  functions  of  the  system.  The  enabling 
products  are  varified  to  be  ready  to  perform  their  intended  functions  when  required  to  support  their  related  end 
product,  or  aggregation  of  end  products.  When  each  set  of  enabling  products  is  developed  using  the  processes 
of  this  Guide,  another  building  block  is  formed  (see  Appendix  F,  Figure  F.3).  Development  of  an  enabling 
product  building  block  is  normally  initiated  after  the  related  end  products  are  fully  defined  and  after  the 
requirements  for  enabling  products  are  identified.  The  building  block  structure  for  an  associated  process  is 
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related  to  only  its  parent  system  building  block  and  does  not  infer  development  of  all  products  related  to  an 
associated  process  for  the  entire  system  structure  (upward  and  downward  in  the  hierarchy  of  building  blocks). 


NOTE  -  Application  of  the  processes  of  this  Guide  to  a  building  block  establishes  the  specified 
requirements  for  the  items  represented  by  black  or  gray  elements  of  Figure  6.1.1.  However,  only 
requirements  (which  most  often  are  not  valid  technical  statements)  are  initially  identified  and  collected 
for  the  enabling  products.  To  represent  this  difference,  the  enabling  product  elements  are  shown  in 
white.  The  processes  of  this  Guide  are  then  applied  to  each  set  of  enabling  products  to  obtain  validated 
technical  requirements  and,  ultimately,  derived  requirements  and  specified  requirements  for  these 
enabling  products. _ 


Examples  of  enabling  products  developed  in  conjunction  with  a  system  are  listed  Table  6. 1.1. 4. 


Table  6.1. 1.4  -  Examples  of  enabling  products  for  each  associated  process 


Associated 

Process 

Examples  of  enabling  products 

Development 

Development  plans  and  schedules,  engineering  policies  and  procedures,  integration  plans 
and  procedures,  information  database,  automated  tools,  analytical  models,  physical 
models,  engineering  management  personnel,  and  connecting  cables  and  other  interface 
structures  not  being  developed  as  separate  end  products. 

Production 

Production  plans  and  schedules,  manufacturing  policies  and  procedures,  manufacturing 
facilities,  jigs,  special  tools  and  equipment,  production  processes  and  materials, 
production  and  assembly  manuals,  measuring  devices,  and  manufacturing  and 
procurement  personnel. 

Test 

Test  plans  (including  test  environment  interactions)  and  schedules,  test  policies  and 
procedures,  test  models,  mass/volume  mockups,  special  tools  and  test  equipment,  test 
stands,  special  test  facilities  and  sites,  measuring  devices,  simulation  or  analytical  models, 
demonstration  and  scale  test  models,  inspection  procedures,  and  test  personnel. 

Deployment 

Deployment  plans  and  schedules,  deployment  policies  and  procedures,  mass/volume 
mockups,  packaging  materials,  special  storage  facilities  and  sites,  special  handling 
equipment,  special  trasportation  equipment  and  facilities,  installation  prodedures, 
installation  brackets  and  cables,  special  transportation  equipment,  deployment 
instructions,  ship  alteration  drawings,  site  layout  drawings,  and  installation  personnel. 

Training 

Training  plans  and  schedules,  training  policies  and  procedures,  simulators,  training 
models,  training  courses  and  materials,  special  training  facilities,  and  trainers. 

Support 

Support  plans  and  schedules,  support  policies  and  procedures,  special  tools  and  repair 
equipment,  maintenance  asistance  modules,  special  services  (for  example,  telephone 
hotline  and  customer  access  lines),  special  support  facilities  and  handling  equipment, 
maintenance  manuals,  maintenance  records  system,  special  diagnostic  equipment  (not  an 
integral  part  of  the  end  product),  and  repair  personnel. 

Disposal 

Disposal  plans  and  schedules,  disposal  policies  and  procedures,  refurbishment  facilities 
and  equipment,  special  disposal  facilities  and  sites,  special  equipment  for  disposal  of  end 
products,  and  disposal  personnel. 

6.1.2  Building  block  roles 

The  building  block  is  used  for:  (1)  identifying  and  assigning  specifications  for  the  system,  end  products,  and 
subsystem  elements;  (2)  managing  interfaces;  (3)  enabling  multidisciplinary  teamwork;  (4)  assessing  risk;  (5) 
structuring  technical  reviews;  and  (6)  cost  collection  and  reporting.  Data  and  document  management  is 
facilitated  by  the  building  block  since  each  system  element  shows  the  source  of  such  data  and  documents.  Data 
and  documents  are  generated  as  work  products  or  deliverables  as  a  result  of  the  technical  efforts  to  develop  each 
system  element.  Likewise,  each  system  element  has  a  work  package  assigned  to  direct  the  team  doing  the 
planned  technical  effort. 
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6.1.2.1 


Specifications 


Specifications  document  the  specified  requirements  that  are  an  output  from  the  Solution  Definition  Process. 
The  building  block  relationships  of  black  and  gray  element  specifications  and  the  white  element  requirements, 
as  well  as  appropriate  interface  specifications,  are  shown  in  Figure  6. 1.2.1. 


System  External 
Interface  Snecifications 


Figure  6.1.2. 1  -  Building  block  role  -  specifications 


NOTES 

1  The  gray  elements  represent  the  system  elements  that  will  be  defined  by  specifications  produced 
by  application  of  the  System  Design  Process  (see  Subsection  4.3),  but  are  not  delivered  as  a  unit. 

2  The  system  specifications  are  the  basis  of  developing  end  product  specified  requirements  and 
associated  sub-processes.  Each  subsystem  specification  is  the  basis  for  development  of  the  next  lower 
layer  building  block  (see  Subsection  6.2). 

Specifications  describe  the  required  characteristics  of  end  products  (black  elements)  or  a  group  of  products 
(gray  elements).  Characteristics  include: 

a)  the  functional  and  performance  requirements; 

b)  interface  requirements; 

c)  the  environments  in  which  the  product(s)  is  required  to  perform  its  functions; 

d)  physical  characteristics  and  attributes; 

e)  the  basis  for  evaluating  test  articles; 

f)  the  methods  for  verifying  compliance; 

g)  the  intenced  uses;  and 

h)  enabling  product  requirements. 
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6.1 .2.1 .1  Stages  of  maturity 

The  specifications  for  the  system,  end  product,  and  subsystem  elements  evolve  through  three  stages  conceptual, 
initial ,  and  established.  Conceptual  specifications  are  used  to  show  feasibility  of  a  higher-level  initial 
specification  (e.g.,  end  product)  and  to  record  the  characteristics  of  notional  products.  Conceptual 
specifications  are  evolved  into  initial  specifications  by  application  of  System  Design  Processes.  Initial 
specifications  are  used  to  direct  lower-layer  building  block  developments  of  subsystems.  The  initial 
specifications  evolve  into  established  specifications  by  application  of  the  System  Design  Processes.  Established 
specifications: 

a)  enable  making  valid  estimates  of  work  and  resources  needed  for  the  next  lower-layer  building  block 
development; 

b)  provide  basis  of  communication  with  and  among  the  development  team,  suppliers,  and  customers; 

c)  provide  guidance  to  testers  for  completing  System  Verification  and  End  Products  Validation  Processes; 

d)  provide  basis  for  negotiation  of  engineering  changes; 

e)  guide  preparation  of  detailed  drawing  or  software  development  file  design  definitions; 

f)  enable  development  of  lower-layer  building  block  specifications  and  solution  definitions,  e.g., 
drawings,  parts  lists,  and  code  lists; 

g)  enable  configuration  management  (control  and  maintenance)  of  solution  definitions  that  satisfy 
technical  requirements;  and 

h)  enable  the  definition  of  logistics  support  for  spares,  replacement  parts,  training,  manuals,  maintenance 
operations,  diagnostic  tools,  and  support  equipment. 

6.1. 2.1 .2  Performance  specifications 

Performance  specifications  are  used  when  it  is  appropriate  to  state  requirements  in  terms  of: 

a)  the  required  results  without  stating  the  method  for  achieving  the  required  results; 

b)  function  (what  is  to  be  accomplished)  and  performance  (how  well  each  function  is  to  be  performed); 

c)  the  environment  in  which  the  product(s)  must  perform  these  functions; 

d)  the  interface  and  interchangeability  characteristics;  and 

e)  the  means  for  verifying  compliance. 

6.1. 2.1. 3  Detail  specifications 

Detailed  specifications  are  used  when  it  is  appropriate  to  state  design  requirements  in  terms  of: 

a)  material  to  be  used; 

b)  how  a  requirement  is  to  be  achieved;  and 

c)  how  a  product  is  to  be  fabricated  or  constructed. 
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NOTE  -  Detail  specifications  are  applicable  guide  creation  of  detailed  drawings  or  the  software 
development  file:  pseudocode  and  software  dictionary. 


6.1. 2.2  Interface  definition 

Interface  specifications  are  essential  in  most  system  development  activities  to  clarify  interdependencies  between 
system  elements  within  the  buiding  block  (internal)  and  other  systems  above,  below,  and  at  the  same  layer  of 
development  (external).  Interface  specifications  are  used  to  define  and  specify: 

a)  physical  and  functional  relationships  between  system  elements,  including  operators; 

b)  functional  requirements  resulting  from  these  relationships;  and 

c)  constraints. 

6.1. 2.3  Multidisciplinary  teamwork 

Another  role  for  the  building  block  is  to  enable  multidisciplinary  teamwork.  A  reference  structure  for  team 
assignment  is  shown  in  Figure  6. 1.2. 3.  Teams  themselves  do  not  ensure  teamwork.  It  is  how  the  teams  are 
integrated  that  is  important,  as  well  as  the  assignment  of  properly  skilled  team  members.  A  system  core  team  is 
usually  composed  of  the  project  technical  manager,  along  with  memebers  to  be  assigned  to  team  lead  positions 
on  end-product  and  associated  process  teams.  An  end-product  team  can  be  the  leaders  from  their  resprective 
subsystem  team.  An  enabling  product  team  can  be  individuals  representing  their  respective  functional 
disciplines.  These  functional  specialists  are  also  assigned  to  subsystem  teams,  as  appropriate.  A  subsystem 
team  is  normally  appropriate  domain  experts  as  well  as  functional  specialists  and  other  required  specialists.  A 
subsystem  team  becomes  the  core  team  for  the  next  lower-layer  building  block  development  of  subsystem  and 
end  products. 


NOTE  -  As  with  the  application  of  any  complex  process,  training  of  all  members  in  the  application  of  the 
concepts  and  practices  in  this  Guide  is  key  to  its  successful  application.  Successful  training  includes  both 
training  that  brings  new  team  members  up  to  speed  and  training  that  refreshes  existing  team  members  on 
the  currently  active  elements  of  the  process  as  the  project  proceeds. _ 


Multidisciplinary  teamwork  ensures  the  accuracy  and  completeness  of  the  evolving  technical  data  package  from 
which  test  articles,  pre-production  prototypes,  and  production  products  are  to  be  manufactured  or  coded. 


Figure  6. 1.2.3  -  Building  block  role  -  teamwork 
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6.1. 2.4 


Risk  management 


Another  role  for  the  building  block  is  to  provide  a  structure  for  assessing  and  managing  risks.  The  risk 
associated  with  arriving  at  the  solution  definition  for  each  end  product  is  a  function  of  the  risk  assigned  to  each 
subsystem  of  the  end  product.  Likewise,  the  risk  associated  with  the  system  of  the  building  block  development 
is  a  function  of  the  end-product  risks  and  the  associated  enabling-product  risks.  The  building  block  shows  the 
relationships  between  subsystems  and  end  products,  and  between  associated  end  products  that  must  be 
considered  in  determining  the  risk  associated  with  each  end  product  development.  Based  on  the  degree  of  risk 
and  the  relationship  among  building  block  elements,  risk  aversion  plans  are  created  and  tracked. 


NOTES 

1  Risk  depends  on  the  probability  of  occurrence  and  its  consequences.  Risk  is  potential  harm  to  the 
project  or  system  under  development.  Risk  is  assessed  for  project,  product,  and  process  aspects  of  the 
system.  This  includes  the  adverse  consequences  of  process  variability.  The  sources  of  risk  include: 
technical  (for  example,  feasibility,  operability,  producibility,  testability,  and  system  effectiveness);  cost 
(for  example,  estimates  and  goals);  schedule  (for  example,  technology/material  availability,  technical 
achievements,  and  milestones);  and  programmatic  (for  example,  resources). 

2  Risk  management  requires  discipline.  Risk  management  is  useful  only  to  the  degree  that  it 
highlights  the  need  for  action,  and  that  action  leads  to  the  problem  being  addressed  quickly  and 
thoroughly.  Moreover,  risk  management  is  continuous.  Things  can  go  wrong  until  the  last  phase  of 
the  project  is  completed. 


6.1. 2.5  Technical  reviews 

Technical  reviews  are  scheduled  and  conducted  during  each  engineering  life-cycle  phase,  as  appropriate,  to 
review  progress  against  plan,  against  the  established  agreement,  and  against  the  applicable  enterprise-based  life- 
cycle  phase  exit  criteria.  They  are  conducted  to  determine  whether  to  continue  the  investment  in  future 
engineering  or  enterprise-based  life-cycle  phases  based  on: 

a)  the  risks  and  costs  associated  with  lower-layer  developments; 

b)  the  maturity  of  the  development  to  date; 

c)  if  requirements  and  technical  plans  being  tracked  are  on  schedule  and  are  achievable  within  existing 
project  constraints; 

d)  resources  required  for  lower-layer  projects;  and 

e)  readiness  to  proceed,  to  include  external  supplier  availability  and  agreement  preparations,  if  applicable. 

The  building  block  also  is  a  convenient  framework  for  technical  reviews  called  out  in  the  agreement  or  the 
engineering  plan.  Two  types  of  reviews  are  conducted  -  Incremen tal  and  System.  Incremental  Reviews  are 
conducted  on  subsystems,  associated  processes  for  related  sets  of  enabling  products,  and  end  products.  Upon 
completion  of  the  incremental  reviews  a  System  Review  (top  element  of  the  building  block)  is  conducted.  The 
typical  order  of  these  reviews  is  shown  in  Figure  6. 1.2. 5. 


M -  Incremental  Reviews  - ► 


Figure  6. 1.2.5  -  Building  block  role  -  technical  reviews 
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The  conduct,  the  reviewing  body,  and  the  presenters  of  specific  technical  reviews  are  planned  in  a  technical 
review  plan  during  the  Planning  Process.  The  team  associated  with  a  specific  review  is  assigned  the  task  of 
creating  and  presenting  the  technical  review.  For  Subsystem  Reviews ,  the  parent  end  product  team  is  typically 
the  reviewing  body.  End  product  team  members  and  team  leads  selected  from  other  associated  process  teams 
make  up  the  reviewing  body  for  the  Associated  Process  Reviews.  These  reviews  can  be  held  as  a  joint  review. 
The  core  team  is  the  reviewing  body  for  the  End  Product  Reviews.  Reviewing  bodies  can  be  supplemented  by 
other  specialists  from  outside  the  project,  as  appropriate  to  meet  technical  review  objectives.  The  reviewing 
body  for  a  System  Review  can  be  designated  in  the  agreement  and/or  in  the  project  plan  or  engineering  plan. 
The  System  Review  can  be  held  along  with  a  project  review  when  intended  to  meet  exit  criteria  for  an 
enterprise-based  life-cycle  phase. 

The  purpose  of  the  Incremental  and  System  Reviews  are  listed  in  Table  6. 1.2. 5 

Table  6. 1.2.5  -  Purposes  of  technical  reviews 


Review 

Purpose 

Subsystem 

To  assess  progress  in  defining  and  satisfying  subsystem  requirements. 

Associated 

Process 

1)  To  assess  progress  and  identify  issues  associated  with  requirements  for  one 
associated  process  or  group  of  associated  processes; 

2)  to  ensure  the  suitability  and  availability  of  the  services  of  enabling  products  when 
they  are  needed. 

End  Product 

To  address  issues  and  demonstrate  required  building  block  development  progress  and 
maturity. 

System 

See  Appendix  E. 

The  technical  reviews  applicable  for  the  engineering  life  cycle  (see  Subsection  6.3)  are  described  in  Appendix 
E.  The  incremental  reviews  are  to  be  completed  prior  to  each  Appendix  E  system  technical  review. 

6.1. 2.6  Cost  collection  and  reporting 

Another  use  of  the  building  block  structure  is  for  collecting  and  reporting  costs  related  to  engineering  life  cycle 
activities.  The  costs  are  incurred  in  each  building  block  system  element  as  development  activities  are  done  in 
accordance  with  assigned  work  packages  generated  during  planning.  The  costs  incurred  include  direct  labor 
costs  associated  with  applying  engineering  process  tasks  for  requirements  definition,  design  definition,  design 
verification,  trade-off  and  effectiveness  analyses,  fabrication,  software  bulk  copying,  technical  reviews,  data 
and  document  generation,  integration,  and  testing. 

Technical  agreement,  planning,  and  control  costs  are  also  collected  and  reported  as  a  part  of  the  development  of 
associated  process  enabling  products.  The  costs  associated  with  a  building  block  system  development  can  be 
easily  summarized  by  rolling  up  the  costs  of  subsystems,  end  products,  and  associated  processes.  When  the 
project  performance  is  tracked  by  an  acquirer,  or  for  internal  control,  using  a  cost  performance  measurement 
system,  cost  and  performance  measurements  are  combined  using  an  earned- value  approach. 

6.2  System  structure  concept 

A  single  building  block  rarely  defines  the  complete  solution  to  acquirer  and  other  stakeholder  requirements.  If  a 
subsystem  requires  further  development,  this  is  done  as  a  subordinate  building  block  development.  Lower- 
Layer  building  block  developments  are  initiated  as  soon  as  definite  contents  of  the  building  block  are 
determined.  The  definite  contents  fo  the  building  block  are  represented  as  end  product  established 
specifications,  initial  subsystem  specificaitons,  interface  specifications,  and  requirements  identified  for 
applicable  enabling  products  of  the  associated  processes.  Building  blocks  are  connected  to  form  the  system 
structure,  or  a  building  block  hierarchy.  The  relationship  among  building  blocks  in  a  hierarchy  is  shown  in 
figure  6.2. 


159 


This  layered  approach  in  the  decomposition  of  building  blocks  continues  until:  (1)  the  end  products  of  a 
building  block  can  be  implemented:  (2)  the  requirements  for  an  end  product  can  be  satisfied  by  an  existing 
product;  or  (3)  the  end  products  can  be  acquired  from  a  supplier.  The  specific  building  block  structure  will  vary 
with  each  system,  based  on  the  number  of  end  products,  the  number  of  subsystems  in  an  end  product,  and  the 
applicable  enabling  products  of  the  associated  processes. 

Layer  N  Building  Block 


Layer  N+l 
Building  Blocks 


Figure  6.2  -  Forming  a  system  structure 


NOTE  -  A  system  structure  serves  as  the  framework  for  the  engineering  of  a  system.  Although 
represented  in  Figure  6.2  as  a  one-to-one  decomposition,  some  cases  can  occur  that  have  multiple 
inheritances  when  the  same  subsystem  or  end  product  can  be  used  several  places  in  the  system 
structure. 


The  specified  requirements  for  a  subsystem  become  the  assigned  requirements  at  the  next  lower  layer  of 
development  (see  Appendix  F).  Each  building  block  can  have  other  stakeholder  requirements  that  are  not 
related  to  the  requirements  that  are  either  assigned  from  above  or  directed  by  users  or  customers. 

6.2.1  Top-down  development 

Figure  6.2  shows  a  view  of  the  layered  development  approach  for  a  project  (also  known  as  a  program  in  some 
domains).  Typically,  the  project  receives  acquirer  requirements  in  a  formal  agreement  (see  Subsection  4.1)  and 
provides  reports  and  delivers  products  in  accordance  with  the  agreement  (see  Subsection  4.2).  Each  project  can 
have  several  lower-layer  building  block  developments.  An  agreement  is  used  for  each  lower-layer  building 
block  development  using  requirements  assigned  from  the  parent  upper-layer  building  block.  Typically,  only 
one  engineering  plan  (see  Subsection  4.2)  is  required  for  the  multiple  layers  of  building  block  developments 
within  a  single  project.  If  an  external  supplier  is  used  for  a  lower-layer  building  block  development,  a  formal 
agreement  is  required. 
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Figure  6.2.1a  is  an  example  system  structure  showing  a  layered  development.  The  top  building  block  contains 
the  end  product  that  must  satisfy  the  primary  user’s  or  customer’s  requirements.  This  top  building  block 
represents  what  is  often  called  the  prime  contractor’s  project.  Two  other  projects  are  shown:  Project  A  and 
Project  B.  The  top  building  block  in  each  of  these  projects  represents  the  top  layer  of  development  for  the 
respective  project,  but  the  second  layer  for  the  prime  contractor’s  project.  Project  A  spawns  two  layers  of 
devlopment,  whereas  Project  B  spawns  multiple  lower-layer  building  block  developments.  The  lines 
connecting  the  layers  reflect  the  specified  requirements  assigned  from  a  parent  building  block  to  its  subordinate 
building  block. 


NOTES 

1 .  It  is  recognized  that  three  approaches  are  practiced  to  engineer  a  system  —  top-down,  bottom-up, 
and  middle-out.  The  approach  in  this  Guide  could  be  considered  both  middle-out  and  top-down.  Since 
the  hierarchy  of  building  blocks  of  Subsection  6.2  starts  in  a  project  that  could  be  anywhere  in  the 
system  structure,  this  could  be  considered  middle-out. 

2.  The  top-down  approach  is  intended  to  flow-down  requirements  so  as  to  ensure  satisfaciton  of  top- 
layer  building  block  project  customer  requirements.  It  is  also  intended  to  take  advantage  of  reuse  and 
off-the-shelf  items  that  satisfy  assigned  requirements  in  order  to  lessen  development  costs  and  shorten 
development  cycle  time.  The  requirements  fo  this  Guide  are  based  on  the  top-down  approach. 

3.  A  bottom-up  approach  to  development  is  normally  not  to  be  used  unless  it  is  ascertained  that  the 
requirements  of  the  top-layer  building  block  project  system  are  not  affected  adversely. 


A  project  applies  the  System  Design  Processes  (see  Subsection  4.3)  to  each  building  block  in  the  project 
boundary  to  develop  the  appropriate  system,  end  product,  and  subsystem  development  specifications  that  are 
defined  to  satisfy  assigned  and  other  stakeholder  requiremnts  related  to  a  single  building  block.  The  products, 
therefore,  do  not  require  further  development.  Project  B’s  second  layer  of  development  has  one  building  block 
that  requires  a  third  layer  of  development,  whereas  the  specifications  of  the  other  building  block’s  end  product 
are  satisfied  by  either  an  off-the-shelf  product  or  a  reuse  product.  Project  B  requires  five  layers  of  development 
to  complete  the  downward  definition  of  end  products  sufficiently  so  that  they  can  be  either  built  or  coded, 
procured  off-the-shelf,  or  reused.  Project  B  relies  on  external  suppliers  for  three  end  products,  one  at  layer  three 
and  two  at  layer  five. 
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Figure  6.2.1a  -  Example  system  structure 


Figure  6.2.1b  shows,  for  Project  B  of  Figure  6.2.1a,  top-down  development  using  the  System  Design  Processes 
(see  Subsection  4.3).  The  inputs  to  each  building  block  include  the  assigned  requirements  from  the  building 
block  above  and  the  other  stakeholder  requirements  that  will  influence  the  building  block  development.  The 
completion  of  the  applicable  planned  technical  efforts  on  each  building  block  is  to  result  in  a  set  of  end  product 
specified  requirements  and  subsystem  initial  specifications,  when  further  development  efforts  are  required. 

The  end  product  specified  requirements  will  be  used  for  End  Product  Verification,  as  well  as  for  procurement  of 
off-the-shelf  or  reuse  end  products,  for  building,  or  for  assembly  and  integration,  as  applicable.  As  the  technical 
efforts  proceed,  design  feedback  is  provided  to  the  parent  building  block  to  ensure  interface  compliance  and 
also  to  ensure  that  design  decisions  do  not  adversely  affect  the  parent  building  block  end  and  enabling  products, 
or  other  subsystems.  Likewise,  the  parent  building  block  provides  any  changes  to  requirements  that  result  from 
other  subsystem  developments,  enabling  product  developments,  or  stakeholder  changes.  Changes  are  passed 
downward  to  lower-layer  building  block  developments,  as  applicable. 
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Figure  6.2.1b  -  Top-down  development 

6.2.2  Bottom-up  realization 

The  previous  subsection  explained  how  end  products  that  make  up  the  system  structure  are  developed,  from  the 
top  down.  Once  specific  end  products  are  defined  sufficiently  by  specifications  so  that  an  off-the-shelf  product 
or  a  reuse  product  can  be  used,  or  so  that  the  end  products  can  be  built  or  coded,  Product  Realization  Processes 
(see  Subsection  4.4)  can  be  initiated.  As  was  shown  in  Figure  6.2.1a,  this  can  occur  at  any  layer  of  the  system 
structure.  However,  the  assembly  or  integration,  verification,  and  validation  of  such  products  occur  from  the 
bottom  up. 

The  bottom-up  realization  of  end  products  is  shown  in  Figure  6.2.2,  again  for  Project  B  (reference  Figure 
6.2.1a).  The  end  products  procured  for  layer  5  (built,  coded,  used  off-the-shelf,  reused,  or  delivered  by  external 
supplier)  are  verified  using  the  End  Products  Validation  Process  (see  Subsection  4.5).  Once  verified,  the  end 
products  are  delivered,  along  with  verification  data,  to  the  parent  building  block,  in  accordance  with  the 
established  agreement.  The  end  product  is  validated  against  its  assigned  requirements,  either  before  delivery  by 
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the  end  product  developer  or  supplier,  or  by  the  layer  4  building  block  developer.  Validation  is  completed  using 
the  End  Products  Validation  Process  (see  Subsection  4.5)  before  being  assembled  or  integrated  with  the  other 
validated  end  products  that  make  up  the  appropriate  composite  end  product  for  the  layer  4  building  block.  This 
composite  end  product  is  then  verified,  and  the  procedure  is  repeated  until  the  project’s  end  product  for  the 
layer  1  building  block  is  delivered  to  the  top-level  developer  in  accordance  with  the  agreement. 
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Figure  6.2.2  -  Bottom-up  realization 

A  key  purpose  of  this  bottom-up  approach  is  to  discover  test-article  variances  and  design  anomalies  at  the 
lowest  layer  of  development  possible  in  order  to  prevent  lower-layer  end  product  defects  from  being  buried  or 
overlooked  and  then  showing  up  during  top-layer  end  product  verification  and  end  product,  or  aggregation  of 
end  products,  validation.  The  System  Design  Processes  are  applied  to  the  affected  building  block  developments 
to  correct  anomalies  uncovered  by  System  Verification  or  End  Products  Validation  Processes  (see  Subsection 
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4.5).  End  products  that  do  not  comply  with  specified  requirements  must  be  remanufactured,  re-coded,  or  re¬ 
procured  to  correct  the  anomaly  or  deficiency  and  so  that  a  corrected  test  article  can  be  verified. 


NOTE — End  product  validation  against  acquirer  requirements  can  be  accomplished  before  delivery,  but 
after  end  product  verification  is  complete,  if  called  for  in  the  agreement.  Otherwise,  the  acquirer 
validates  the  delivered  end  product  prior  to  assembly  or  integration  with  other  end  products  to  make  up 
the  composite  end  products  appropriate  to  the  building  block.  The  aggregation  of  end  products  might 
also  need  to  be  validated. 


6.3  Engineering  life  cycle  concept 

Each  product  within  a  system  structure  has  its  own  life  cycle.  The  product  line  it  represents  is  developed  and 
produced  to  meet  acquirer  requirements  and  is  then  inserted  into  the  marketplace  either  to  satisfy  an  established 
agreement  or  through  marketing.  Following  insertion,  there  is  growth  stage  where  the  product  becomes  a  viable 
product  in  the  marketplace.  This  is  followed  by  a  maturity  stage  where  the  product  is  no  longer  in  growing 
demand,  where  competitor  products  take  part  of  the  market,  or  where  the  product  fails  to  achieve  its  market 
potential  and  demand  levels  off  and  starts  to  decline.  Finally,  the  sales  of  the  product  decline,  and  the  product  is 
phased  out  and  is  no  longer  marketed  or  distributed.  All  products  undergo  this  life  cycle. 


NOTE — The  product  life  cycle  is  the  one  generally  defined  in  project  management  books.  This  life 
cycle  is  the  driver  for  the  two  other  life  cycles,  in  that  new  products  must  be  developed,  or  that  legacy 
products  are  improved,  to  create  new  business  and  profitability  to  an  enterprise,  or  to  keep  systems 
competitive  to  meet  external  threats  to  an  enterprise  or  nation. _ 


The  processes  of  this  Guide  are  applicable  at  any  point  in  a  product’s  life  cycle.  In  the  early  stages  of  a 
product’s  life  cycle,  the  processes  for  engineering  a  system  are  applied  to  bring  the  system,  or  a  portion  thereof, 
into  realization.  System  products  are  then  produced  and  transitioned  into  operations  where  products  are  used 
and  supported  and  during  which  operators  and  maintainers  are  trained.  As  products  are  used  and  as  design 
anomalies  or  desired  product  improvements  are  identified,  the  processes  of  this  Guide  are  applied  to  reengineer 
the  products.  Finally,  during  product  retirement,  the  processes  of  this  Guide  are  applied  to  correct  any  enabling 
product  design  anomaly  for  the  retirement  or  disposal  process. 

The  layers  of  development  shown  in  Figures  6.2.1a  and  6.2.2  are  directly  correlated  with  a  set  of  engineering 
life-cycle  phases  during  which  the  processes  of  this  Guide  are  applied.  The  engineemg  life-cycle  phases  are 
described  in  Table  6.3.  These  phases  are  grouped  as  follows:  (1)  Conception,  consisting  of  the  Pre- System 
Definition  Phase;  (2)  Creation,  consisting  of  the  System  Definition,  Subsystem  Design,  and  Detailed  Design 
phases;  and  (3)  Realization,  consisting  of  the  End  Product  Physical  Integration,  Test,  and  Evaluation  phase. 

Figure  6.2.1a  shows  application  of  the  phases  related  to  Conception  and  Creation  for  the  top-down 
development.  Figure  6.2.2  shows  application  of  the  phase  related  to  bottom-up  Realization.  Appendix  B 
describes  how  these  groups  of  phases  are  used  in  individual  enterprise-based  life-cycle  phases  to  incrementally 
evolve  the  system  products  before  implementing  the  utilization  phases  of  the  enterprise-based  life  cycle,  or 
before  continuing  the  utilization  enterprise-based  life-cycle  phase  in  which  the  system  products  were  improved 
using  the  activities  of  the  engineering  life-cycle  phases. 


NOTE — The  engineering  life  cycle  applies  in  the  research  and  development  stage  of  a  product’s  life 
cycle,  but  it  also  applies  during  any  product  life-cycle  phase  or  enterprise-based  life-cycle  phase  when  it 
is  needed  as  a  result  of  engineering  or  reengineering  decisions,  (see  Appendix  B) _ 
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Table  6.3  Engineering  life-cycle  phases 


PHASE 

DESCRIPTION  OF  ACTIVITIES 

Pre- System 
Definition 

This  is  the  start-up  phase  of  the  engineering  life  cycle.  The  Technical  management  Processes,  as 
applicable,  are  applied  to  plan  a  technical  effort,  or  refine  the  technical  effort  described  by 
existing  plans,  that  is  consistent  with  an  established  agreement 

System  Design  Processes  are  applied,  as  appropriate,  to  the  top-layer  building  block  of  a  project 
to  determine  the  best  system  concepts  to  satisfy  acquirer  requirements,  or  to  refine  a  previously 
selected  concept,  or  legacy  system,  established  in  a  prior  enterprise-based  life-cycle  phase.  A  set 
of  initial  specifications  for  the  system  and  selected  end  products  of  the  system  concept  is 
defined,  as  appropriate,  and  technology  requirements,  risks,  and  other  constraints  are  identified. 
Before  progressing  to  the  next  phase,  appropriate  incremental  technical  reviews  and  a  system 
concept  review  are  completed. 

System 

Definition 

System  Design  Processes,  and  appropriate  Technical  management  and  Technical  Evaluation 
Processes,  are  applied  to  the  top-layer  building  block  of  a  project  to  establish  specified 
requirements  for  the  end  products  and  to  define  initial  specifications,  including  interface 
specifications,  for  subsystems  of  each  end  product,  and  to  identify  enabling  product  requirements 
to  enable  and  end  product  to  meet  functionality  requirements  during  development,  production, 
test,  deployment,  training,  support,  and  disposal,  as  applicable.  Identified  high  technical  risk 
areas  are  mitigated  during  this  phase.  Before  progressing  to  the  subsystem  design  phase, 
appropriate  incremental  technical  reviews  and  a  system  definition  review  are  completed. 

Subsystem 

Design 

System  Design  Processes,  and  appropriate  Technical  Management  and  Technical  Evaluation 
Processes,  are  applied  to  the  building  blocks  at  the  second  layer  of  the  project  to  establish 
specified  requirements  for  the  end  products  and  to  define  initial  specifications,  including 
interface  specifications,  for  subsystems  of  each  end  product,  and  to  identify  enabling  product 
requirements  to  enable  an  end  product  to  meet  functionality  requirements  during  development, 
production,  test,  deployment,  training,  support,  and  disposal,  as  applicable.  Identified  high 
technical  risk  areas  for  subsystem  end  products  and  enabling  products  are  averted  during  this 
phase.  Before  progressing  to  the  detailed  design  phase,  appropriate  incremental  technical 
reviews  and  a  system  preliminary  design  review  are  completed. 

Detailed 

Design 

System  Design  Processes,  and  appropriate  Technical  Management  and  Technical  Evaluation 
Processes,  are  applied  to  the  building  blocks  at  the  third  and  lower  layers  of  the  project  to 
establish  specified  requirements  and  detailed  drawings  or  documents,  as  appropriate,  for  the  end 
products  and  to  define  initial  specifications,  including  interface  specifications,  for  subsystems  of 
each  end  product  that  requires  further  development,  and  to  identify  enabling  product 
requirements  to  enable  an  end  product  to  meet  functionality  requirements  during  development, 
product,  test,  deployment,  training  support,  and  disposal,  as  applicable.  Identified  high  technical 
risk  areas  for  lower-layer  end  products  and  enabling  products  are  averted  during  this  phase. 

Before  progressing  to  the  next  lower-layer  detailed  design  effort,  appropriate  incremental 
technical  reviews  and  a  system  detailed  design  review  are  completed  on  the  applicable  building 
block  elements.  When  an  end  product  design  can  be  fulfilled  by  buying,  building,  or  reuse, 
development  of  that  end  product  is  complete.  Prior  to  progressing  to  the  next  phase  of  the 
engineering  life  cycle,  test  readiness  and  production  readiness  technical  reviews  are  completed. 

End 

Product 
Physical 
Integration, 
Test  and 
Evaluation 

End  products  are  obtained  from  suppliers,  acquirers  ( in  the  case  of  customer- furnished  items), 
are  off-the  shelf,  or  are  fabricated,  based  on  completed  detailed  design  specifications, 
documents,  or  drawings.  The  Implementation  process,  Technical  Management  Processes,  and 
Technical  Evaluation  Processes  are  applied  to  validate  end  products  obtained,  to  assemble  or 
integrate  validated  end  products,  and  to  verify  that  composite  end  products  satisfy  specified 
requirements.  The  Transition  to  Use  Process  is  applied  to  deliver  the  verified  end  products  to  the 
acquirer  of  the  next  layer  up  in  accordance  with  the  established  agreement.  Then,  the 
implementation  Process,  the  Technical  Management  Processes,  the  Technical  Evaluation 
Processes,  and  the  Transition  to  Use  Process  are  applied,  as  appropriate,  to  successive  upper- 
layer  building  blocks  until  delivery  of  the  end  products  and  enabling  products  required  in  the 
agreement  that  establish  the  project. 
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Appendix  A  -  Glossary  (normative) 


For  the  purposes  of  this  Guide,  the  following  definitions  apply: 

acquirer:  An  enterprise,  organization,  or  individual  that  obtains  a  product  (good  or  service)  from  a  supplier. 


NOTES 

1  The  acquirer  can  be  a  customer  or  user  of  a  desired  system  product,  or  can  be  a  developer  obtaining 
a  lower  layer  product  in  the  system  hierarchy  from  another  vendor  or  a  developer  in  the  role  of  supplier. 

2  An  acquirer  is  a  type  of  stakeholder. 


agreement:  An  arrangement,  not  necessarily  contractual,  between  two  parties  (an  acquirer  and  a  supplier)  that 
defines  the  tasks  to  be  performed,  the  items  to  be  delivered,  the  acceptance  criteria  to  be  applied  to  delivered 
items,  and  other  requirements  affecting  the  development  or  procurement  of  system  products. 

assign:  Designate  a  function,  product,  process,  or  other  item  as  accountable  for  a  particular  purpose. 


NOTES 

1  The  terms  allocate  or  partition  are  used  in  some  domains  to  denote  this  concept. 

2  The  “assign”  relationship  can  be  in  various  forms:  a)  requirement  to  function,  b)  requirement  to  product  or  process,  c) 
requirement  to  interface,  d)  function  to  product  or  process,  e)  function  to  external  entity  (e.g.,  the  operator),  or  f)  requirement  to 
external  entity  (e.g.,  external  system). 


associated  processes:  Processes  that  enable  one  or  more  end  products  to  be  put  into  service,  maintained  in 
service,  or  retired  from  service. 

building  block:  A  representation  of  the  conceptual  framework  of  a  system  that  is  used  for  organizing  the 
requirements,  work,  and  other  information  associated  with  the  engineering  of  a  system.  An  element  in  the 
structured  decomposition  of  the  system. 

configuration  management:  A  management  process  for  establishing  and  maintaining  consistency  of  a 
product’s  performance,  functional,  and  physical  attributes  with  its  requirements,  design,  and  operational 
management  information  throughout  its  life.  Reference:  ANSI/EIA-649. 

constraint:  (1)  A  restriction,  limit,  or  regulation  imposed  on  a  product,  project,  or  process.  (2)  A  type  of 
requirement  or  design  feature  that  cannot  be  traded  off. 

customer:  An  individual,  organization,  or  enterprise  that:  (1)  commissions  the  engineering  of  a  system;  (2)  is  a 
prospective  purchaser  of  the  end  products  of  a  system,  or  portions  thereof;  or  (3)  is  an  acquirer  of  a  product. 

deliverable:  An  item  agreed  to  be  delivered  to  an  acquirer  as  specified  in  an  agreement.  This  item  can  be  a 
document,  a  hardware  item,  a  software  item,  a  service,  or  any  type  of  work  product. 

derivative  system:  A  special  type  of  precedented  system  derived  from  a  previously  operational  system  through 
the  use  of  major  elements,  but  whose  requirements  have  been  modified  to  meet  new  objectives. 

derived  requirement:  (1)  A  requirement  that  is  further  refined  from  a  primary  source  requirement  or  a  higher- 
level  derived  requirement.  (2)  A  requirement  that  results  from  a  design  decision  for  a  logical  or  physical 
solution  representation. 

developer:  An  enterprise  or  organization  that  performs  the  process  requirements  of  this  Guide. 

development:  The  action  by  which  a  set  of  requirements  is  translated  into  a  solution  definition  for  a  set  of 
products  that  satisfy  stakeholders. 

document:  A  collection  of  data,  regardless  of  the  medium  on  which  it  is  recorded,  that  generally  has 
permanence  and  can  be  read  by  humans  or  machines. 
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NOTE — Documentation  is  an  instance  of  a  document  or  a  collection  of  documents. 


effectiveness  analysis:  An  assessment  of  how  well  a  product  associated  with  an  alternative  logical,  physical,  or 
design  solution  is  expected  to  perform  or  operate,  given  an  anticipated  usage  scenario. 

enabling  product:  Item  that  provides  the  means  for  a)  getting  an  end  product  into  service,  b)  keeping  it  in 
service,  or  c)  ending  its  service. 


NOTE — Enabling  products  are  related  to  the  associated  processes:  development,  production,  test,  deployment,  training, 
support,  and  disposal. 


end  product:  The  portion  of  a  system  that  performs  the  operational  functions  and  is  delivered  to  an  acquirer. 

end  product  validation:  Confirmation  by  examination  and  provision  of  objective  evidence  that  the  specific 
intended  use  of  an  end  product  (developed  or  purchased),  or  an  aggregation  of  end  products,  is  accomplished  in 
an  intended  usage  environment. 


NOTES 

1  The  key  difference  between  end  product  validation  and  end  product  verification  is  that  end  product 
validation  answers  the  question:  Does  the  delivered  end  product  conform  to  the  validated  input  acquirer 
requirements,  certification  criteria,  or  acceptance  criteria,  as  applicable?  End  product  verification 
answers  the  question:  Does  the  output  end  product  comply  to  the  output  specified  requirements  from 
which  the  end  products  were  built,  coded,  procured,  or  assembled  and  integrated? 

2  End  product  validation  is  used  to  demonstrate  that  the  product  developed  or  purchased  satisfies  the 
validated  acquirer  requirements  in  the  context  of  its  intended  use. 

3  Validation  against  other  stakeholder  requirements,  generally,  is  not  required.  These  requirements 
generally  act  as  constraints  on  either  the  solution  or  the  process  by  which  a  solution  is  generated. 
Constraints  on  solutions  will  show  up  in  specifications  to  which  an  end  product  is  built,  coded,  or 
assembled,  and  then  verified  against.  Process  constraints  will  be  evaluated  during  management  reviews 
or  in  management  reports. 

4  Validated  is  used  to  designate  the  corresponding  status. _ 


end  product  verification:  Confirmation  by  examination  and  provision  of  objective  evidence  that  the  specified 
requirements  to  which  an  end  product  is  built,  coded,  or  assembled  have  been  fulfilled. 


NOTES 

1  End  product  verification  is  used  to  demonstrate  that  the  specified  requirements  (specifications) 
generated  by  the  developer  and  used  to  build,  code,  or  assemble  the  end  product  have  been  satisfied. 

2  Verified  is  used  to  designate  the  corresponding  status. _ 


engineering  life  cycle:  A  sequence  of  phases  that  evolves  an  instance  of  a  system  from  a  concept  to  a  set  of 
products  consistent  with  the  exit  criteria  established  for  an  enterprise-based  life-cycle  phase. 

engineering  plan:  The  plan  for  implementing  the  processes  for  engineering  a  system.  The  engineering  plan 
reflects  an  integrated  technical  effort  that  balances  all  factors  associated  with  meeting  life  cycle  requirements. 

enterprise:  The  entity  that  has  governance  over  a  set  projects,  or  over  organizations  in  which  projects  are 
carried  out. 

enterprise-based  life  cycle:  The  incremental  progress  of  a  system  from  conception  through  disposal,  marked  by 
management-established  milestones  with  assigned  exit  criteria. 

environment:  (1)  The  natural  conditions  (weather,  climate,  ocean  conditions,  terrain,  vegetation,  dust,  etc.)  and 
induced  conditions  (electromagnetic  interference,  heat,  vibration,  etc.)  that  constrain  the  design  definitions  for 
end  products  and  their  enabling  products.  (2)  External  factors  affecting  an  enterprise  or  project.  (3)  External 
factors  affecting  development  tools,  methods,  or  processes. 

function:  A  task,  action,  or  activity  performed  to  achieve  a  desired  outcome. 
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functional  requirement:  A  requirement  that  defines  what  system  products  must  do  and  their  desired  behavior 
in  terms  of  an  effect  produced,  or  an  action  or  service  to  be  performed. 


NOTES 

1  An  example  of  a  behavior  is  “system  switches  from  standby  mode  to  run  mode;”  an  example  of  an 
effect  produced  is  “cause  an  alert  signal;”  an  example  of  an  action  or  service  to  be  performed  is  “signal 
opens  valve.” 

2  A  functional  requirement  can  include  the  actor  that  is  to  perform  the  function,  the  function  to  be 
performed,  and,  if  appropriate,  the  object  acted  upon.  In  addition,  this  information  can  be  complemented 
by  a  statement  of  the  environment  within  which  the  function  is  performed,  the  conditions  that  cause  the 
function  to  start,  the  performance  requirements  associated  with  that  function,  and  the  conditions  that 
cause  the  function  to  terminate. 


information  database:  A  repository  that  provides  a  capacity  to  maintain  work  products  and  outcomes  from 
implementation  of  the  processes  for  engineering  a  system  in  a  controlled  manner. 


NOTE — This  database  provides  the  basis  for  controlled  maintenance  of  the  information  needed  by  the 
multidisciplinary  teams  and  management  to  efficiently  and  effectively  accomplish  their  assigned  tasks. 
It  typically  contains  the  requirements,  configurations  of  a  system  (past,  current,  and  planned),  and  all 
analyses  and  test  results.  This  database  allows  for  traceability,  supports  the  validation  and  verification 
tasks,  is  essential  for  change  management,  and  provides  information  to  support  decision  making. _ 


interface  requirement:  A  requirement  that  defines  the  conditions  of  interaction  between  items. 


NOTES 

1  Interface  requirements  include  both  logical  and  physical  interfaces.  They  include,  as  necessary, 
physical  measurements,  definitions  of  sequences  of  energy  or  information  transfer,  and  all  other 
significant  interactions  between  items. 

2  There  are  interfaces  between  a  system  and  things  external  to  the  system,  and  between  elements 
within  a  system.  The  latter  include,  but  are  not  limited  to,  interfaces  between  the  end  products  and  their 
operators  or  maintainers,  the  interfaces  between  items  that  make  up  an  end  product,  and  interfaces 
between  an  end  product  and  enabling  products  of  the  associated  processes. 

3  For  example,  communications  interfaces  involve  the  movement  and  transfer  of  data  and  information 

within  the  system,  and  between  the  system  and  its  environment.  Proper  evaluation  of  communications 
requirements  involves  definition  of  both  the  structural  components  of  communications  (e.g.,  bandwidth, 
data  rate,  distribution,  etc.)  and  content  requirements  (what  data/information  is  being  communicated, 
why  it  is  being  moved  among  the  system  components,  and  the  criticality  of  this  information  to  system 
functionality). _ 


layer  of  development:  (1)  A  level  of  abstraction  as  it  relates  to  the  system  structure  made  up  of  building  blocks. 
(2)  A  level  of  system  decomposition. 

method:  Techniques  that  support  implementation  of  process  tasks. 


NOTE —  A  method  is  the  “how”  of  each  task.  Methods  have  the  following  attributes:  a)  thought 
patterns  or  approaches;  b)  knowledge  base;  c)  rules  and  heuristics;  d)  structure  and  order;  and  e) 
notation. 


multidisciplinary  teamwork:  The  cooperative  application  of  all  appropriate  disciplines  by  people  functioning 
as  a  team  to  achieve  solutions  that  balance  the  contributions  of  the  disciplines  effectively. 

normative:  That  portion  of  a  Guide  or  specification  that  governs  implementation. 
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NOTE  -  A  standards  document  usually  contains  three  kinds  of  material:  (1)  The  standard  itself 
(normative  part);  (2)  explanatory  material  to  help  the  user  understand  the  standard  (informative 
part);  and  (3)  other  material  concerning  the  administration  of  the  standard  and  the  sponsoring 
organization  (administrative  part).  The  explanatory  material  is  contained  in  Notes  or  “informative 
annexes.”  Conformance  to  a  standard  is  judged  solely  on  the  basis  of  the  normative  material  in  the 
standards  document. 


operational  scenario:  A  sequence  of  events  expected  during  operation  of  system  products.  Includes  the 
environmental  conditions  and  usage  rates  as  well  as  expected  stimuli  (inputs)  and  responses  (outputs). 

performance  requirement:  A  requirement  that  defines  how  well  the  system  products  are  required  to  perform  a 
function,  along  with  the  conditions  under  which  the  function  is  performed. 

precedented:  An  end  product  that  is  a  legacy  product  undergoing  modification  or  a  product  that  the  enterprise 
both  has  the  expertise  to  make  and  has  similar  products  already  in  the  market  place. 

process:  A  set  of  interrelated  tasks  that,  together,  transform  inputs  into  outputs. 

product:  (1)  An  item  that  consists  of  one  or  more  of  the  following:  hardware,  software,  firmware,  facilities, 
data,  materials,  personnel,  services,  techniques,  and  processes.  (2)  A  constituent  part  of  a  system. 

project:  A  development  effort  consisting  of  both  technical  and  management  activities  for  the  purpose  of 
engineering  a  system. 


NOTE —  For  the  purposes  of  this  Guide,  project  and  program  are  synonymous. 


prototype:  A  model  (physical,  electronic,  digital,  analytical,  etc.)  of  a  product  built  for  the  purpose  of:  a) 
assessing  the  feasibility  of  a  new  or  unfamiliar  technology;  b)  assessing  or  mitigating  technical  risk;  c) 
validating  requirements;  d)  demonstrating  critical  features;  e)  verifying  a  product;  f)  validating  a  product;  g) 
determining  enabling  product  readiness;  h)  characterizing  performance  or  product  features;  or  i)  discovering 
physical  principles. 

requirement:  (1)  Something  that  governs  what,  how  well,  and  under  what  conditions  a  product  will  achieve  a 
given  purpose.  (2)  Normative  elements  that  govern  implementation  of  this  Guide,  including  certain  documents 
such  as  agreements,  plans,  or  specifications. 

requirements  validation:  Confirmation  by  examination  that  requirements  (individually  and  as  a  set)  are  well 
formulated  and  are  usable  for  intended  use. 


NOTES 

1  See  Table  C.25  for  what  constitutes  “well  formulated.” 

2  There  are  five  types  of  requirements  validation  in  this  Guide  stated  in  Sub-processes  25 
through  29. 


risk:  (1)  A  measure  combining  the  uncertainty  of  reaching  a  goal  with  the  consequences  of  failing  to  reach  the 
goal.  (2)  The  probability  of  suffering  injury  or  loss. 

risk  aversion:  The  act  of  averting  risk.  Averting  risk  can  be  through  various  means:  mitigation,  avoidance, 
transfer,  or  acceptance. 

risk  management:  An  organized  process  for  identifying  and  assessing  risks,  and  for  implementing  means  to 
avoid  them  or  mitigate  their  effect  if  they  occur. 

specification:  A  document  that  contains  specified  requirements  for  a  product  and  the  means  to  be  used  to 
determine  that  the  product  satisfies  these  requirements. 

stakeholder:  An  enterprise,  organization,  or  individual  having  an  interest  or  a  stake  in  the  outcome  of  the 
engineering  of  a  system. 
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NOTES 

1  Examples  of  stakeholders  are  acquirer,  user,  customer,  manufacturer,  installer,  tester,  maintainer, 
executive  manager,  project  manager,  and  all  other  personnel  having  a  stake  in  the  development  or 
outcome  of  the  engineering  of  a  system.  The  enterprise  as  a  corporation  or  agency  and  the  general 
public  are  also  stakeholders. 

2  An  acquirer  (see  definition  above)  is  a  specific  instance  of  a  stakeholder  and  is  individually 
acknowledged  since  the  acquirer  is  a  principal  in  establishing  the  acquirer- supplier  agreement. 

3  All  stakeholders  other  than  the  acquirer  are  referred  to  as  “other  stakeholders”. _ 


stakeholder  requirement:  A  requirement  that  represents  what  stakeholders  of  a  system  need  or  expect  of  the 
system  products. 

standard:  A  document  that  establishes  engineering  and  technical  requirements  for  products,  processes, 
procedures,  practices,  and  methods  that  have  been  decreed  by  authority  or  adopted  by  consensus. 

subsystem:  A  grouping  of  items  that  perform  a  set  of  functions  within  a  particular  end  product. 

supplier:  Provides  a  product  (either  end  products,  enabling  products,  or  both)  or  a  group  of  products  to  an 
acquirer.  The  supplier  (external  or  internal  to  the  acquirer’s  organization)  can  be  a  vendor  that  has  a  product 
that  does  not  need  development,  or  a  developer  that  must  develop  the  desired  system  product  or  products. 

system:  An  aggregation  of  end  products  and  enabling  products  to  achieve  a  given  purpose. 

system  technical  requirement:  A  requirement  derived  from  one  or  more  stakeholder  requirements  and  stated 
in  technical  terms. 

technical  performance  measurement  (TPM):  The  technique  of  predicting  the  future  value  of  a  key  technical 
parameter  of  the  higher-level  end  product  under  development,  based  on  current  assessments  of  products  lower 
in  the  system  structure. 


NOTES 

1  Involves  the  continuing  verification  of  the  degree  of  anticipated  and  actual  achievement  for  technical 
parameters.  Confirms  progress  and  identifies  variances  that  might  jeopardize  meeting  a  higher-level  end  product 
requirement.  Assessed  values  falling  outside  established  tolerances  indicate  a  need  for  evaluation  and  corrective 
action. 

2  Key  characteristics  of  TPM  are: 

a)  Achievement  to  Date — present  achieved  value  of  the  technical  parameter  based  on  estimates  or  actual 
measurement; 

b)  Current  Estimate — the  value  of  the  technical  parameter  predicted  to  be  achieved  by  the  end  of  the 
technical  effort  with  remaining  resources  (including  schedule  and  budget); 

c)  Technical  Milestone — a  point  where  TPM  evaluation  is  accomplished  or  reported; 

d)  Planned  Value  Profde — the  projected  time -phased  achievement  projected  for  the  technical  parameter  from 
the  beginning  of  the  development  or  as  replanned  as  a  result  of  a  corrective  projection; 

e)  Tolerance  Band — an  envelope  containing  the  Planned  Value  Profile  and  indicating  the  allowed  variation 
and  projected  estimation  error; 

f)  Objective — the  goal  or  desired  value  at  the  end  of  the  technical  effort; 

g)  Threshold — the  limiting  acceptable  value  that,  if  not  met,  would  jeopardize  the  project; 

h)  Variation — the  difference  between  the  planned  value  and  the  achievement-to-date  value. _ 


technical  review:  An  event  at  which  the  progress  of  the  technical  effort  is  assessed  relative  to  its  governing 
plans  and  technical  requirements. 

test  article:  An  item  built,  constructed,  coded,  or  otherwise  implemented,  for  checking  conformance  to 
specified  requirements  or  for  checking  validation  against  acquirer  requirements  for  the  item. 

traceability:  The  ability  to  identify  the  relationship  between  various  artifacts  of  the  development  process,  i.e., 
the  lineage  of  requirements,  the  relationship  between  a  design  decision  and  the  affected  requirements  and  design 
features,  the  assignment  of  requirements  to  design  features,  the  relationship  of  test  results  to  the  original  source 
of  requirements. 
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unprecedented:  A  specific  end  product  that  is  not  known  a  priori ,  or  the  enterprise  has  limited  experience  in 
developing  this  type  of  system. 

user:  Individual,  organization,  or  enterprise  that  uses,  applies,  or  operates  system  products, 
validation:  See  end  product  validation  and  requirements  validation. 
verification:  See  end  product  verification. 
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Appendix  B  -  Enterprise-based  Life  Cycle  (normative) 


The  various  commercial  and  non-commercial  enterprises,  within  widely  diverse  domains,  have  similar 
enterprise-based  lifecycles,  and  generally  exist  for  the  same  purpose.  That  purpose  is  to  incrementally  develop 
systems  and  control  passage  from  one  increment  to  another  so  as  to  reduce  risk,  control  costs,  and  provide  and 
maintain  system  products  that  will  be  competitive  and  provide  user  safe  satisfaction  throughout  the  life  cycle. 

Each  enterprise-based  life  cycle  is  characterized  by  distinct  phases  marked  by  established  exit  criteria  and 
management  reviews  to  ensure  that  the  exit  criteria  are  satisfied  prior  to  making  a  decision  on  whether  or  not  to 
approve  progress  to  the  next  phase  or  sequence  of  phases,  or  to  make  modifications  or  improvements  to 
maintain  competitiveness.  Although  the  various  enterprise-based  life  cycles  may  have  different  named  phases, 
and  different  phase  and  life  cycle  time  periods,  most,  if  not  all,  have  these  five  distinct  functional  phases:  (1) 
assessment  of  opportunities,  (2)  investment  decision,  (3)  system  concept  development,  (4)  subsystem  design 
and  pre-deployment,  and  (5)  deployment/installation,  operations,  support,  and  disposal. 

B.1  Relationship  to  engineering  life-cycle  phases 

Figure  B.l  shows  five  typical  phases  of  an  enterprise-based  life  cycle  and  the  use  of  appropriate  engineering  life 
cycle  activities  to  meet  the  exit  criteria  for  the  enterprise-based  life-cycle  phases. 


System 

Subsystem 

Deployment, 

Concept 

Design  & 

Operations, 

Develonment 

Pra-Denlm/ment 

Sunnort  &  Disnosal 

Advanced  Technology 
Prototype 


Improvements, 
As  Necessary 


Simulation,  Physical 
or  Functional  Prototype 


Pre-Production  Prototype, 
Production  Runs 


Figure  B.l  -  Enterprise-based  life-cycle  phases 
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NOTES 

1  It  is  during  the  investment  decision  phase  that  a  commercial  or  subsystem  supplier  organization 
typically  prepares  a  proposal  in  response  to  a  competitive  solicitation,  when  such  governs  an  enterprise’s 
activities.  In  other  organizations,  such  as  government  agencies,  solicitation  and  proposal  activities  can 
occur  before  any  of  the  above  phases  when  competition  is  deemed  appropriate. 

2  Enterprise-based  life  cycles  tend  to  be  unique  to  an  enterprise,  and  are  subdivided  by  different 
phases  that  depend  on  the  needs  of  the  enterprise.  These  are  generally  based  on  the  enterprise’s  own 
external  environment.  They  are  established,  for  example,  in  response  to  market  cycles,  government 
agency  directives,  or  fiscal  considerations.  They  are  not  generally  based  on  engineering  efforts  required 
for  a  system  development  (or  portions  thereof),  but  on  entry  and  exit  criteria  to  meet  internal  or 
customer-  driven  milestones. 

3  The  key  message  of  Figure  B.l  is  that  appropriate  engineering  life  cycle  process  activities  are 
completed  to  meet  the  exit  criteria  of  each  enterprise-based  life-cycle  phase,  regardless  of  the  name  or 
purpose  of  the  phase. 


B.2  Product  evolution 

During  the  early  phases  of  this  generic  life  cycle,  various  levels  of  system  products  are  developed.  For  instance, 
during  the  first  phase  (assessment  of  opportunities),  a  simulation-produced  prototype  can  be  used  to  identify, 
qualify,  and  select  new  or  improved  system  and  business  opportunities. 

During  the  second  phase  (investment  decision),  a  physical  or  functional  prototype  can  be  developed  to 
understand  a  solution  so  that  determination  can  be  made  whether  to  continue  with  the  development  and  so  that 
project  plans  are  produced  in  preparation  for  transition  to  system  development.  For  competitive  developments, 
a  bid  or  no  bid  can  be  determined  and  a  proposal  can  be  developed,  if  necessary. 

During  the  third  phase  (system  concept  development),  an  advanced  technology  prototype  can  be  developed, 
including  one  sufficiently  operational  to  access  performance  and  cost  factors  and  to  identify  and  reduce  critical 
risk  factors. 

The  forth  phase  (subsystem  design  and  pre-deployment)  produces  a  pre-production  prototype,  which  will  be 
used  for  verifications  and  validations  and  acceptance  by  the  acquirer,  and  required  production  volume  of  end 
products  and  enabling  products  for  deployment  or  installation. 

The  last  phase  (deployment/installation,  operations,  support,  and  disposal)  is  where  the  system  products  are 
deployed  or  installed  for  operation  and  various  operational,  maintenance,  and  disposal  support  provided,  as 
required.  During  this  last  phase,  reengineering  is  often  necessary  to  keep  the  products  competitive  and  useful. 

If  needed,  the  processes  of  this  Guide  are  applied  while  using  the  appropriate  engineering  life-cycle  phases. 

B.3  Life  cycle  considerations 

Cost  is  an  important  criterion  for  both  making  a  decision  to  develop  a  certain  system  and  buying  that  system. 
Various  heuristics  attribute  that  from  60  to  80  percent  of  the  life  cycle  cost  of  a  system  is  experienced  in  the 
operations  and  support  phase.  It  is  essential,  therefore,  that  focus  be  on  ways  to  reduce  such  costs  during  the 
earlier  phases  of  the  enterprise-based  life  cycle.  It  is  important  to  treat  cost,  especially  pertaining  to  associated 
processes,  as  an  independent  variable  while  making  trade-off  analyses. 
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Appendix  C  -  Process  Task  Outcomes  (informative) 


This  Appendix  provides  an  informative  set  of  representative  tasks  and  their  expected  outcomes  for  the  thirty- 
three  sub-processes  of  Section  4. 

C.1  Acquisition  and  Supply  task  outcomes 

Table  C.l  -  Sub-process  1  (Supply  Process  -  Product  Supply) 


a)  Representative 
tasks 

Expected  outcomes 

a)  Assess  acquisition 
request,  offer,  or 
directive 

The  capability  of  the  enterprise,  organization,  project,  or  team  to  provide  a  system,  or 
portion  thereof,  that  meets  acquisition  document  requirements  within  the  stated 
constraints  and  the  enterprise  strategic  plan  and  business  strategy,  or  within  the  project 
plan  and  constraints,  or  within  the  team  charter,  as  applicable,  is  determined. 

Includes,  as  appropriate: 

1)  engineering  and  other  applicable  technical  and  project  plans  that  allow 
determination  of  engineering  and  management  tasks,  costs,  and  schedules, 
resource  requirements,  and  technical  capabilities  and  capacities  (invoke 
applicable  Planning  Process  tasks); 

2)  decision  whether  to  work  with  the  acquirer  to  provide  the  desired  system,  or  a 
portion  thereof,  based  on  establishment  enterprise  criteria  or  on  project  or  team 
capability; 

3)  resolution  of  added  or  changed  requirements  and  areas  of  concern; 

4)  preparation  and  submission  of  an  appropriate  technical  and  cost  response  in 
accordance  with  acquisition  requirements,  enterprise  business  strategy,  and 
enterprise  policies  and  procedures,  or  with  project  plans,  policies,  and  directives. 

b)  Negotiate 
agreement 

A  satisfactory  agreement  is  established  based  on  the  bounds  determined  by,  as 

applicable: 

1)  applicable  legal,  regulatory,  policies,  procedures,  and  practices  that  will  affect 
negotiation  strategy  or  conduct; 

2)  the  type  of  agreement  to  be  negotiated; 

3)  negotiation  strategy; 

4)  conditions  identified  from  the  plans  for  the  procurement  work  effort  that  could 
affect  negotiations  and  agreement  performance; 

5)  constraints  identified  from  the  plans  for  the  procurement  work  effort  that  could 
affect  negotiations  and  agreement  performance. 

c)  Record  agreement 

Established  agreement  is  captured  in  a  form  and  medium  appropriate  to  the  effort. 

d)  Implement 
agreement 

A  project  established  and  processes  (including  replanning,  as  necessary)  activated  to 
complete  the  requirements  of  the  agreement. 

e)  Deliver  product  and 
other  deliverables 
per  agreement 

Agreement  requirements  satisfied  by  the  delivery  of  required  products  and  other 
deliverables  in  accordance  with  agreement  instructions. 
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Table  C.2  -  Sub-process  2  (Acquisition  Process  -  Product  Acquisition) 


Representative  tasks 

Expected  outcomes 

a)  Prepare  acquisition 
requests,  offers,  or 
directives 

Acquisition  documents,  as  applicable  to  the  technical  effort,  prepared  to  include: 

1)  plans  to  be  provided  to  suppliers,  as  applicable; 

2)  purpose  of  the  acquisition,  the  essential  requirements  to  be  met,  the  products  to 
be  delivered  by  a  supplier,  and  the  operational  concept  and  expected  operational 
environment  for  each  product,  as  applicable; 

3)  what  the  products  to  be  delivered  must  be  able  to  do;  how  well  the  products  must 
perform;  desired  characteristics  of  the  products,  constraints,  and  other  essential 
product  attributes;  management  concerns  including  line  of  authority,  financial 
management,  and  reporting;  and  requirements  that  can  affect  the  cost,  schedule, 
and  risk  in  accomplishing  the  work  effort  or  delivery  of  the  product; 

4)  concerns  such  as  cost  and  schedule  that  can  constrain  the  work  effort  or  product, 
and  states  whether  or  not  the  concern  can  be  traded  off; 

5)  expected  tasks  or  work  to  be  done  by  the  supplier; 

6)  the  data  and  other  work  products  to  be  delivered,  including  form,  format,  and 
schedule. 

b)  Evaluate  supplier 
response 

Supplier  or  suppliers  selected  that  will  do  the  agreed-to  work  and  provide  the  desired 
products,  as  appropriate. 

c)  Make  offer  or 
provide  directive 

Offer  made  or  directive  provided  to  the  selected  supplier  or  suppliers. 

d)  Negotiate  agreement 

A  satisfactory  agreement  established  based  on  the  bounds  determined  by,  as 

appropriate: 

1)  applicable  legal,  regulatory,  policies,  procedures,  and  practices  that  will  affect 
negotiation  strategy  or  conduct; 

2)  the  type  of  agreement  to  be  negotiated; 

3)  negotiation  strategy; 

4)  conditions  identified  from  the  plans  for  the  procurement  work  effort  that  could 
affect  negotiations  and  agreement  performance; 

5)  constraints  identified  from  the  plans  for  the  procurement  work  effort  that  could 
affect  negotiations  and  agreement  performance. 

e)  Record  agreement 

Established  agreement  is  captured  in  a  form  and  medium  appropriate  to  the  effort. 

f)  Accept  delivered 
products 

Installed  or  delivered  system  products  validated  as  satisfying  the  user,  customer,  or 
assigned  requirements,  and  other  applicable  certification  or  acceptance  criteria. 

Table  C.3  -  Sub-process  3  (Acquisition  Process  -  Supplier  Performance) 


Representative  tasks 

Expected  outcomes 

a)  Define  supplier 
relationships 

The  type  of  supplier  support  required,  level  of  participation,  procedures  and  criteria 
for  selection  and  control,  procedures  for  participation,  as  appropriate,  on  developer’s 
multidisciplinary  teams,  and  an  appropriate  acquirer-supplier  agreement  are 
established. 

b)  Participate  on 
product  teams 

Agreed-to  procedures  for  participation  of  supplier  personnel  on  developer 
multidisciplinary  product  teams  and  for  participation  of  developer  personnel  on 
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supplier  multidisciplinary  product  teams  are  implemented. 

c)  Monitor  product 
metric  data 

Supplier  performance  against  product  metrics  established  in  the  agreement  is 
determined.  Invoked  the  applicable  tasks  in  the  Assessment  Process. 

d)  Flow-down  changes 
in  requirements  of 
operational  concept 

Assurance  made  that  all  requirement  and  operational  concept  changes  affecting  the 
supplier’s  project  have  been  properly  communicated  to  the  supplier. 

e)  Control  requirement 
changes 

All  changes  approved  to  functional  and  performance  requirements  and  to  constraints, 
made  by  the  supplier,  that  would  affect  the  developer’s  project  or  other  related 
projects  or  products.  Approved  changes  have  been  appropriately  distributed  and 
implemented. 

f)  Assess  progress 
against  requirements 

Progress  against  assigned  requirements  included  in  the  agreement  and  as  changed  by 
established  change  procedures  is  determined.  Required  technical  reviews  completed. 
Invoked  applicable  tasks  of  the  Assessment  Process. 

g)  Validate  products 
received 

Assurance  made  that  delivered  products  satisfy  assigned  requirements  and  approved 
changes.  Resolution  of  identified  variations  resulting  from  validation  of  the  delivered 
product  is  complete.  Invoked  the  applicable  tasks  of  the  End  Products  Validation 
Process. 

Table  C.4  -  Sub-process  4  (Planning  Process  -  Process  Implementation  Strategy) 


Representative  tasks 

Expected  outcomes 

a)  Identify 
stakeholders 

Intended  users  or  customers  and  other  stakeholders  who  will  have  an  interest  or  stake 
in  the  outcome  of  the  project  are  established. 

b)  Identify  applicable 
documents 

Applicable  source  and  technical  documents  and  the  requirements  therein  that  could 

affect  the  project  effort  are  identified  and  acquired,  including: 

1)  the  scope  and  purpose  of  both  the  project  and  products  to  be  developed  or 
reengineered; 

2)  stated  purpose  of  the  products,  expectations  of  the  stakeholders,  expected 
benefits  to  stakeholders,  as  well  as  the  goals  and  objectives  of  the  system,  or 
portion  thereof,  to  be  developed  or  reengineered; 

3)  enterprise  policies,  priorities,  and  constraints  on  funding,  personnel,  facilities, 
manufacturing  capability  and  capacity,  and  critical  resources  that  will  affect 
accomplishing  the  requirements  and  goals  of  the  source  and  technical 
documents;  and 

4)  (a)  applicable  processes,  standards,  and  specifications;  (b)  core  enterprise 
technologies;  (c)  risks  to  business  growth  by  new  project;  (d)  must-win  criteria; 

(e)  net  cost  targets;  (f)  methods  of  resource  allocation;  (g)  how  work  and  changes 
will  be  authorized;  (h)  how  information  will  be  captured;  (i)  how  work  packages 
will  be  formed  and  controlled  (j)  scope  and  procedures  for  trade-off  analyses, 
effectiveness  analyses,  and  risk  management  based  on  enterprise  goals  and 
planning  baselines. 

c)  Identify  associated 
process  approaches 

How  development  of  enabling  products  associated  with  production,  test, 
deployment/installation,  and  logistics  processes  will  be  implemented  is  determined. 

d)  Identify  applicable 
life-cycle  phases 

Applicable  enterprise-based  life-cycle  phases  (see  Appendix  B.2),  the  expected  work 
product  outputs  and  management  reviews,  and  the  relevant  exit  criteria  for  each 
applicable  enterprise-based  life-cycle  phase,  including  level  of  product  maturity 
expected,  level  of  acceptable  risk,  management  review  concerns,  and  documentation 
requirements,  are  determined. 
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e)  Identify  and  define 
technical  process 
and  project 
integration 

How  the  applicable  processes  of  this  Guide  will  be  integrated  with  each  other  and 
with  other  processes  specified  in  enterprise  and  agreement  documents,  and  which 
internal  and  external  projects  that  will  be  involved  and  how  they  will  be  integrated 
are  determined. 

f)  Identify  and  define 
progress  against 
assessment 

Required  reporting  requirements,  specific  product  and  process  metrics  to  be  used, 
how  and  when  metrics  will  be  collected  and  by  whom,  and  how  progress  will  be 
assessed  are  determined. 

g)  Prepare  the  process 
implementation 
strategy 

A  process  implementation  strategy  document  based  on  the  integrated  results  of  the 
outcomes  of  the  above  tasks  is  prepared. 

Table  C.5  -  Sub-process  5  (Planning  Process  -  Technical  Effort  Definition) 


Representative  tasks 

Expected  outcomes 

b)  Identify  project 
requirements 

The  following  are  determined: 

1)  Specific  requirements  include  (a)  work  that  the  supplier  is  required  to 
accomplish,  (b)  functions  of  the  system,  or  portion  thereof,  to  be  furnished, 
engineered,  or  improved;  how  well  the  functions  are  to  be  performed;  any 
required  physical  characteristics;  and  the  operations  concept,  (c)  data  to  be 
delivered  and  when,  (d)  budget  and  schedule  requirements,  and  (e)  other 
technical  requirements  provided  in  acquirer-supplied  planning  documents; 

2)  Other  stakeholders  who  have  or  who  will  have  requirements  or  expectations  with 
respect  to  the  work  to  be  accomplished  or  the  system  to  be  provided  (for 
example,  local,  national,  or  international  government  agencies;  persons  living  or 
working  in  the  areas  near  where  system  products  will  be  used  or  where  products 
will  be  developed  and  produced;  commercial  or  military  competitors;  and 
employees  involved  with  the  project); 

3)  Potential  conflicts  between  the  acquirer-supplier  agreement  (proposed  or  final), 
the  process  implementation  strategy,  and  enterprise  policies  and  procedures,  core 
technologies,  and  capacities; 

4)  Specific  constraints  and  any  conflict  between  the  process  implementation 
strategy  and  the  agreement  (proposed  or  final)  with  respect  to  development, 
production,  test,  deployment,  support,  or  disposal  of  the  system  products  to  be 
delivered,  or  the  training  of  personnel  required  to  operate  and  maintain  the 
products. 

c)  Establish 
information 
database 

The  types  and  quantity  of  data  and  schema  and  other  information  that  will  have  to  be 
recorded  and  maintained,  as  well  as  access  and  security  requirements,  are 
determined;  a  database  that  can  securely  retain  and  make  available  project 
information,  as  required,  is  established. 

d)  Define  risk 
management 
strategy 

The  following  are  determined:  (1)  how  the  technical  risk  areas  of  the  technical  effort 
will  be  identified  and  tracked;  and  (2)  the  appropriate  risk  aversion  approaches  based 
on  the  acceptable  levels  of  risk  specified  in  the  agreement  or  in  enterprise  policies 
and  procedures. 

e)  Define  product  and 
process  metrics 

The  following  are  defined;  (1)  product  metrics  by  which  the  quality  of  the  product  is 
to  be  evaluated;  (2)  process  metrics  by  which  the  efficiency  and  effectiveness  of  the 
tasks  of  the  technical  effort  are  to  be  evaluated;  and  (3)  frequency  and  methods  by 
which  product  and  process  metrics  are  to  be  collected. 

f)  Establish  cost 

Rigorous  cost  goals  (ownership,  acquisition,  operating,  support,  and  disposal)  to  be 
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objectives 

used  in  trade-off  analyses  are  established. 

g)  Identify  technical 
performance 
measures 

The  following  are  determined:  (1)  technical  objectives  related  to  success  of  the 
system,  or  portion  thereof,  [e.g.,  measures  of  effectiveness  (MOEs)  by  which  the 
user,  customer,  or  acquirer  will  measure  satisfaction  or  acceptance];  and  (2)  key 
performance  parameters  that  will  receive  management  focus  and  are  to  be  tracked 
using  Technical  Performance  Measurement  (TPM)  procedures. 

h)  Identify  applicable 
tasks 

The  following  are  identified:  (1)  key  events  of  the  project  (e.g.,  technical  reviews, 
physical  integration,  major  test,  product  and  process  verifications,  and  end  product 
validation)  established  by  input  planning  documents;  (2)  entry  and  exit  completion 
criteria  for  each  event;  and  (3)  tasks  required  for  meeting  the  entry  and  exit  criteria  of 
each  event  and  for  accomplishing  each  applicable  process. 

NOTE  -  The  following  structure  of  tasks  can  be  helpful  for  accomplishing 
scheduling  staffing  determination,  and  resources  required: 

1  Key  events  required  to  meet  technical  requirements  (e.g.,  test  and 
technical  review). 

2  Primary  tasks  related  to  accomplishing  entry  and  exit  criteria  of  each 
key  event  (e.g.,  define  stakeholder  requirements  and  prepare  engineering 
drawings). 

3  Support  tasks  that  enable  the  staff  accomplishing  primary  tasks  to  meet 
their  objectives  (e.g.,  provide  resources,  equipment,  facilities,  acquire 
appropriately  skilled  personnel  for  accomplishing  primary  tasks,  and  arrange 
travel). 

4  Management  tasks  required  to  direct,  monitor,  review,  and  approve  the 
primary  and  support  tasks  (e.g.,  serve  as  chair  of  a  technical  review,  and 
review  and  approve  documents  for  transmittal  to  the  customer). 

i)  Identify  methods 
and  tools 

The  following  are  determined:  (1)  appropriate  methods  for  accomplishing  identified 
tasks,  or  groups  of  tasks  of  each  applicable  process;  (2)  required  automated  tools;  (3) 
required  specialized  facilities  and  equipment;  and  (4)  training  requirements. 

j)  Establish 

technology  insertion 
approaches 

The  applicable  or  potential  technology  constraints  are  identified  and  the  approach  for 
conducting  parallel  technology  developments,  and  planned  technology  insertions  are 
established. 

Table  C.6  -  Sub-process  6  (Planning  Process  -  Schedule  and  Organization) 


Representative  tasks 

Expected  outcomes 

a)  Develop  event- 
based  schedule 

The  key  events  for  the  technical  effort  associated  with  applicable  enterprise-based 
life-cycle  phases,  related  applicable  tasks  to  each  event,  and  the  completion  criteria 
for  each  task  and  an  event  are  developed  and  documented. 

b)  Develop  calendar- 
based  schedule 

The  calendar  date  that  each  key  event  will  be  completed  or  expected  to  be  completed; 
the  planned  start  and  completion  time  for  accomplishment  of  each  task  (primary, 
support  and  management);  and  the  dependency  relationships  between  tasks,  between 
tasks  and  events,  and  between  events  and  other  events  are  developed  and 
documented. 

c)  Identify  resource 
requirements 

The  material  resources,  facilities,  and  equipment  required  to  complete  each  scheduled 
primary,  support,  and  management  tasks  are  determined,  and  the  date  such  resources 
are  required  is  specified. 

d)  Define  staffing 

needs  and  discipline 

The  following  are  determined:  (1)  personnel  needs  by  discipline  and  performance 
level  to  complete  scheduled  primary,  support,  and  management  tasks,  and  the  date 
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needs 

each  staffing  need  is  required;  (2)  internal  and  external  supplier  training  needs  and 
schedules  to  achieve  required  proficiencies;  and  (3)  risk  to  the  project,  if  adequate 
staffing  is  not  available. 

e)  Define  team  and 
organizational 
structure 

(1)  The  multidisciplinary  teams  needed  to  carry  out  the  planned  technical  efforts  and 
produce  required  work  products  are  formed  within  enterprise  and  project  resource 
constraints;  (2)  The  composition  of  teams  by  functional  and  disciplinary  membership 
that  are  organized  to  support  specific  system  product  development  is  established;  (3) 
The  names  of  staff  members  assigned  to  each  team  are  established;  (4) 

Responsibilities  and  authority  of  teams  and  team  members  are  defined;  and  (5)  Roles, 
responsibilities,  authority  and  boundaries  for  each  team  are  established. 

Table  C.7  -  Sub-process  7  (Planning  Process  -  Technical  Plans) 


Representative  tasks 

Expected  outcomes 

a)  Develop 

Engineering  Plan 

An  efficient  and  economical  means  of  implementing  the  processes  for  engineering  a 
system  is  defined  and  documented.  It  answers  the  following  questions: 

1)  What  is  the  general  problem  to  be  solved? 

2)  What  is  the  benefit  to  the  acquirer  (enterprise  perspective)? 

3)  What  is  the  application  context  of  the  general  problem  to  be  solved? 

4)  What  is  the  boundary  of  the  general  problem  to  be  solved,  denoting  what  can 
be  controlled  by  the  developer  (inside)  and  what  influences  the  development 
and  is  influenced  by  the  development  but  not  controlled  by  the  developer 
(outside)? 

5)  What  are  the  required  inputs  and  outputs? 

6)  What  are  the  influencing  factors  and  constraints? 

7)  How  are  the  system  concerns,  as  appropriate,  of  reliability,  availability, 
maintainability,  security,  safety,  health  factors,  survivability,  electro¬ 
magnetic  compatibility,  radio  frequency  management,  and  human  factors 
being  considered  and  included? 

8)  What  processes  and  tasks  must  be  accomplished? 

9)  How  will  each  process  be  accomplished? 

10)  What  resources,  methods,  and  tools  are  necessary  to  accomplish  the  tasks  of 
each  process? 

11)  How  will  the  required  resources  and  tools  be  acquired? 

12)  What  is  the  organizing  structure? 

13)  How  will  the  organization  be  staffed  and  managed? 

14)  What  are  key  intermediate  events  leading  to  project  completion,  and  how 
will  their  occurrence  be  determined? 

15)  When,  where,  and  by  whom  will  tasks  and  events  be  completed? 

1 6)  What  are  the  risks  involved?  How  will  risks  be  managed? 

17)  What  are  the  completion  criteria  for  the  process  tasks? 

1 8)  What  are  the  entry  and  exit  criteria  for  reaccomplishing  each  process? 

19)  How  will  project  completion  be  determined? 

NOTES 

1  The  engineering  plan  usually  covers  one  or  more  phases  of  the 
enterprise-based  life  cycle  and  the  applicable  phases  of  the  engineering  life 
cycle. 

2  The  engineering  plan  is  to  cover  process  applications  within  the 
engineering  life  cycle  to  meet  the  exit  criteria  of  the  applicable  enterprise- 
based  life-cycle  phases,  as  consistent  with  the  acquirer-supplier  agreement 
and  the  extent  of  the  project  conducted  within  an  enterprise. 

b)  Develop  Risk 
Management  Plan 

Documentation  of  the  tasks  to  be  accomplished  by  project  teams  and  analysis  for 
identification  of  potential  risks,  characterization  and  prioritization  of  identified  risks, 
aversion  of  risks,  and  tracking  and  control  of  risks,  and  communication  of  risk  status 
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are  defined  and  documented. 

c)  Develop  Technical 
Review  Plan 

Tasks  to  be  accomplished  to  implement  required  technical  reviews  and  a  detailed 
description  for  each  review  are  developed  and  documented  to  include:  (1)  a  check  list 
for  tasks  to  be  accomplished,  (2)  entrance  and  exit  criteria,  (3)  review  schedule,  (4) 
documentation  requirements,  (5)  distribution  list  for  technical  data  package,  (6) 
participants,  and  (7)  responsibilities  of  participants. 

d)  Develop  Validation 
Plans 

The  tasks  to  be  accomplished  and  the  resources  to  be  allocated  and  scheduled  for 
validating  that:  (1)  the  system  technical  requirements,  logical  representations,  and 
derived  technical  requirements  are  well  formulated  (see  Sub-process  25)  and  conform 
to  their  respective  sources,  and  (2)  the  products  received  from  suppliers,  or  delivered 
to  an  acquirer,  conform  to  the  user,  customer,  or  assigned  requirements  associated 
with  the  end  product  are  defined  and  documented. 

e)  Develop 

Verification  Plans 

The  tasks  to  be  accomplished  and  the  resources  to  be  allocated  and  scheduled  for 
verifying  that:  (1)  the  selected  and  characterized  physical  solution  description 
satisfies  the  assigned  system  technical  requirements,  logical  representations,  and 
derived  technical  requirements  (2)  end  products  satisfy  their  specified  requirements, 
and  (3)  enabling  products  will  be  ready  when  required  to  provide  life  cycle  support  to 
their  respective  end  products  are  defined  and  documented. 

f)  Develop  Other 
Applicable  Plans 

The  tasks  to  be  accomplished  to  complete  required  control  activities  or  other  design 
activities  such  as  design- to-cost,  Technical  Performance  Measurement,  technology 
insertion,  safety,  security,  human  factors  engineering,  and  maintenance  reliability 
(see  Appendix  D  for  others),  as  required  in  an  agreement  or  by  enterprise  policies  and 
procedures,  are  defined  and  documented. 

Table  C.8  -  Sub-process  8  (Planning  Process  -  Work  Directives) 


Representative  tasks 

Expected  outcomes 

a)  Develop  work 
packages 

The  work  required,  input  sources,  schedules,  budget,  and  reporting  requirements  to 
implement,  execute,  and  control  the  work  are  defined  and  documented. 

b)  Generate  work 
authorizations 

Approval/disapproval  of  work  packages  is  assigned,  and  work  authorizations  are 
documented. 

Table  C.9  -  Sub-process  9  (Assessment  Process  -  Process  Against  Plans  and  Schedules) 


Representative  tasks 

Expected  outcomes 

a)  Identify  events, 
tasks,  and  process 
metrics  for 
monitoring 

The  events  and  tasks  that  must  be  monitored,  as  well  as  the  metrics  that  will  be  used 
to  assess  progress  against  plans  and  schedules,  are  identified.  The  applicable 
expected  values  for  each  progress  metric  are  established. 

b)  Collect  and  analyze 
process  metric  data 

Results  from  completion  of  required  tasks  and  events,  and  process  metrics  data  are 
determined  and  tracked. 

c)  Compare  process 
metrics  data  against 
plans  and  schedules 

The  following  are  determined:  (1)  completion  of  required  tasks  and  events,  (2) 
variances  of  metrics  from  expected  values,  (3)  progress  variances  from  plans  and 
schedules,  (4)  technical  areas  requiring  management  or  team  attention,  and  (5)  cost 
and  schedule  risk. 

d)  Implement  required 
changes 

The  cost  effective  changes  to  correct  variances  and  needed  changes  to  plans  and 
schedules,  and  required  changes,  revised  work  directives,  and  updated  plans  to  reflect 
approved  changes  and  management  decisions  are  identified,  approved,  and 
implemented. 
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Table  C.10  -  Sub-process  10  (Assessment  Process  -  Progress  Against  Requirements) 


Representative  tasks 

Expected  outcomes 

a)  Identify  product 
metrics  to  be 
monitored 

Product-related  metrics,  and  their  expected  values,  that  will  affect  the  quality  of 
the  product  and  provide  information  of  the  progress  toward  satisfying 
user/assigned  requirements,  other  stakeholder  requirements,  and  derived 
requirements  are  identified  and  documented. 

b)  Collect  and  analyze 
product  and  metrics 
data 

The  following  are  determined,  as  appropriate:  (1)  analyzed,  estimated,  or  measured 
values  of  key  performance  parameters  at  predetermined  events  (e.g.,  simulation 
and  prototype  tests),  (2)  compliance  to  applicable  requirements,  (3)  levels  of 
technical  risks,  (4)  marginal  cost  benefit  of  performance  beyond  requirements,  (5) 
degree  of  customer  satisfaction  and  public  acceptance,  and  (6)  effect  of  a  key 
performance  parameter  status  on  related  end-user  products. 

c)  Record  rationale  for 
decisions  and 
assumptions  made 

The  following  are  determined,  as  applicable:  (1)  satisfaction  of  alternatives  based 
on  recommendations  and  effects  of  trade-off  and  effectiveness  analyses  and  (2) 
assumptions  associated  with  decisions  made  during  requirements  definition, 
solution  definition,  trade-off  analyses,  effectiveness  analyses,  verifications,  and 
validations. 

d)  Compare  results 
against 
requirements 

The  following  are  determined,  as  applicable:  (1)  satisfaction  of  technical 
requirements,  (2)  progressive  maturity  of  the  system,  or  portion  thereof,  being 
engineered/reengineered,  (3)  variances  from  expected  values  from  Technical 
Performance  Measurements,  and  (4)  variations  from  requirements  resulting  from 
end  product  verifications  and  end  product  validations. 

e)  Identification  and 
Implementation  of 
Required  Changes 

The  following  are  identified,  evaluated,  and  implemented,  as  applicable:  (1) 
alternative  corrective  actions  to  mitigate  out-of-tolerance  Technical  Performance 
Measurements,  (2)  other  changes  to  be  implemented  so  that  products  will  meet 
requirements,  (3)  recommended  user/assigned,  other  stakeholder,  or  technical 
requirement  changes,  and  (4)  implementation  of  revised  specifications  and 
configuration  baselines  that  reflect  approved  changes  and  management  decisions. 

Table  C.ll  -  Sub-process  11  (Assessment  Process  -  Technical  Review) 


Representative  tasks 

Expected  outcomes 

a)  Identify  technical 
review  objectives 
and  requirements 

The  following  are  identified  and  documented:  (1)  purpose  and  objectives  of  the 
review,  (2)  agenda  requirements,  (3)  tasks  to  be  completed  at  each  required  review, 

(4)  entrance  and  exit  requirements,  (5)  documentation  requirements,  (6)  distribution 
requirements,  and  (7)  responsibilities  of  the  review  participants. 

b)  Determine  progress 
against  event-based 
plan 

The  satisfaction  of  entrance  requirements  to  the  review  are  determined  and 
documented. 

c)  Establish  technical 
review  board, 
agenda  and  speakers 

For  each  review,  the  following  are  established:  (1)  persons  who  will  participate  in  the 
review,  (2)  chairpersons,  (3)  secretary,  (4)  reviewers  of  the  presentation,  (5)  agenda 
that  meets  review  requirements  and  ensures  that  all  required  tasks  are  completed,  and 
(6)  members  of  the  design  team  that  will  prepare  the  data  package,  and  prepare  the 
presentation,  prepare  material  for  distribution  at  the  review,  make  presentations, 
answer  questions,  and  accomplish  task  close  out  action  items. 

d)  Prepare  technical 
review  package  and 
presentation 
materials 

Comprehensive  read-ahead  material  is  prepared  that  includes  sufficient  information 
so  that  technical  board  members  can  understand  the  design  and  participate  effectively 
in  the  review.  Review  team  responsibilities,  agendas,  plans,  and  expectations  from 
the  review  are  defined  and  documented.  A  comprehensive  set  of  presentation 
materials  that  describe  the  assigned  design  topics  and  that  satisfy  review  objectives  is 
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prepared. 

e)  Facilitate  resolution 
of  emerging  issues 

Emerging  issues  identified  and  resolved  prior  to  the  review. 

f)  Conduct  technical 
review 

The  following  are  assessed  by  the  review:  (1)  maturity  of  system,  or  portion  thereof, 
being  engineered,  (2)  progress  according  to  plans  and  requirements,  (3)  risks  and 
variances  in  cost  schedule,  and  performance,  and  (4)  readiness  to  proceed  with  the 
next  phase  of  development.  Action  items  required  to  meet  review  objectives  are 
generated,  recorded  and  assigned. 

g)  Close-out  review 

The  following  are  completed  for  review  close-out:  (1)  preparation  and  distribution  of 
minutes  that  include  purpose,  time,  place,  attendees,  decisions,  action  items,  due  date, 
and  persons  responsible  for  resolving  action  items,  (2)  resolution  of  action  items,  and 
(3)  sign  off  by  chairperson. 

Table  C.12  -  Sub-process  12  (Control  Process  -  Outcomes  Management) 


Representative  tasks 

Expected  outcomes 

a)  Capture  process 
outcomes 

The  following  are  recorded  in  the  information  database:  (1)  the  outputs  of  the 
technical  processes  implemented  in  the  engineering  of  a  system,  (2)  the  methods, 
tools,  models,  and  metrics  used,  (3)  recommendations,  decisions,  assumptions,  and 
effects,  (4)  lessons  learned,  and  (5)  other  data  that  allows  traceability  of  requirements. 

b)  Perform 

configuration 

management 

The  configuration  of  the  products  is  documented  and  made  available.  The  following 
is  realized:  (1)  product  configuration  is  known  and  reflected  in  product  information, 

(2)  beneficial  product  changes  are  effected  without  adverse  consequences,  (3)  change 
is  managed  from  the  first  implemented  phase  during  system  design,  (4)  information 
that  will  be  needed  to  make  later  decisions  on  products  is  captured,  (5)  consistency 
between  a  product  and  information  about  the  product,  and  (6)  capability  to 
distinguish  between  product  versions  or  builds. 

NOTE  -  ANSI/EIS-649  can  be  used  in  conjunction  with  this  Guide,  for 

configuration  management. 

c)  Perform  change 
management 

Traceability  of  change  is  maintained  and  controlled,  including  source  of  the  change, 
processing  methods,  approvals,  and  implementations  in  accordance  with  the  Change 
Management  Plan. 

d)  Perform  interface 
management 

System  internal  and  external  interfaces  are  maintained  and  controlled,  including 
completion  of  interface  definition,  assessments  of  compatibility,  changes,  and 
coordinations  and  approvals  in  accordance  with  the  Interface  Management  Plan. 
Interfaces  are  managed,  ensuring  that:  (1)  all  internal  and  external  functional  and 
physical  (including  human)  interfaces  for  a  building  block  are  identified,  defined, 
assigned,  documented,  and  managed,  (2)  building  block  design  definitions  are 
compatible  in  terms  of  form,  fit,  and  function,  and  (3)  interface  changes  affecting  the 
building  block  and  affected  by  the  building  block  (see  Section  6)  are  controlled  to 
prevent  adverse  consequences. 

e)  Perform  risk 
management 

Potential  risks  are  identified,  characterized  and  prioritized,  and  properly  averted, 
tracked  and  controlled.  Risk  status  is  communicated  in  progress  reports,  in  proposals, 
and  at  technical  reviews,  in  accordance  with  the  Risk  Management  Plan.  A  clear 
view  of  future  risks  is  provided,  better  decision  making  is  enabled,  and  problems  are 
prevented  from  occurring  -  but  if  they  do  occur,  a  plan  exists  to  mitigate  the  effect  of 
the  problem. 

NOTES 

1  Risk  is  always  present  in  an  engineering  or  reengineering  project.  Sources 
of  risk  include  the  tendency  of  acquirers  to:  (1)  desire  products  of  a  system  that 
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are  intended  for  technical  accomplishment  near  the  limits  of  the  state  of  the  art 
(performance),  (2)  push  for  delivery  of  system  products  as  soon  as  possible  to 
meet  an  imminent  market  opportunity  or  threat,  and  (3)  limit  funding  available. 
Additionally,  risks  come  from  both  internally  and  externally  imposed  constraints 
(e.g.,  resource,  capacities,  environmental  conditions,  and  reuse). 

2  The  major  sources  of  risk  are  programmatic,  schedule,  political,  financial 
and  technical.  Risks  are  greater  when  planning,  control,  resources  and  time  are 
inadequate.  Risks  are  also  greater  when  information  is  not  available  for  decision¬ 
making,  or  when  the  information  is  too  much,  too  little,  irrelevant,  or  inaccurate. 

f)  Perform  data  and 
document 
management 

Data  and  documents  are  maintained  and  controlled,  including  development  support, 
handling  and  storage,  and  required  technical  data  and  document  delivery  in 
accordance  with  the  Data  Management  Plan.  Data  and  document  management 
includes  capturing  data  and  documents  generated  during  implementation  of  the 
processes  of  this  Guide,  and  generating  and  maintaining  an  evolving  technical  data 
package.  A  typical  data  package  includes:  (1)  a  buy- to  description  (e.g.,  detail 
specifications  and/or  final  drawings),  (2)  a  build-to  description,  (3)  design 
documentation,  (4)  engineering  changes,  deviations,  and  waivers,  and  (5)  enabling 
product  descriptions. 

Build-to  descriptions  include:  (1)  models,  drawings,  and  specifications,  (2) 
production  planning,  (3)  tool  design,  (4)  bill  of  materials,  and  (5)  statistical  process 
control  plan. 

NOTE  -  Multidisciplinary  teamwork  is  essential  to  ensure  the  accuracy  and 

completeness  of  technical  manuals  and  the  technical  data  package. 

g)  Manage  information 
database 

Relevant  data  and  information  are  maintained  and  controlled  for  the  project, 
including  inputs  and  outputs  of  control  process  tasks  and  ensuring  back-ups,  if 
applicable,  of  digital  databases.  Relevant  data  includes: 

1 .  Inputs  and  outputs  of  technical  process  activities: 

a)  work  products  (e.g.,  specifications,  drawings,  and  code  lists); 

b)  archival  data  (e.g.,  decisions  made  [including  rationale],  assumptions, 
lessons  learned,  changes,  and  empirical  data); 

c)  stakeholder  requirements  (e.g.,  technical  objectives,  constraints,  and 
interfaces); 

d)  requirement,  functional,  and  physical  architectures; 

e)  physical  models  developed  (e.g.,  prototypes,  breadboards,  brassboards,  and 
mock  ups); 

f)  simulation  model  outputs  and  assumptions; 

g)  metrics  (e.g.,  cost  and  technical  performance  measures); 

h)  planning  documents  (e.g.,  engineering  plan  and  technical  event  plan); 

i)  technologies. 

2.  Process  models  used  for: 

a)  analysis  of  problem  (analysis  of  requirements  and  analysis  of  functions) 

(e.g.,  Quality  Function  Deployment,  behavior,  and  time); 

b)  solution  definition  (synthesis)  (e.g.,  for  design); 

c)  validation  and  verification; 

d)  system  analysis  (e.g.,  for  trade-off  analyses,  risk  analyses,  and  effectiveness 
analyses); 

e)  control  (e.g.,  interfaces,  data,  configurations,  schedules,  costs,  product 
performance,  reviews,  and  assessments). 

3.  Tools  used: 

a)  automated  tools  (e.g.,  traceability,  analysis,  and  design); 

b)  validation  and  verification  tools; 

c)  trade-off  analysis  support  tools; 

d)  communication  tools;  and 
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e)  status  reporting/projection  tools. 

h)  Manage  and  track 
requirements 

The  following  are  maintained  and  controlled:  (1)  input  requirements  (acquirer  and 
other  stakeholder),  system  technical  requirements,  logical  solution  representations, 
physical  solution  representations,  derived  technical  requirements,  and  specified 
requirements,  (2)  validation  results,  (3)  requirement  changes  resulting  from  resolution 
of  variances,  and  (4)  changes  made  to  requirements  through  formal  change 
procedures  from  Configuration  Management,  Change  Management,  and  Interface 
Management  tasks. 

Table  C.13  -  Sub-process  13  (Control  Process  -  Information  Dissemination) 


Representative  tasks 

Expected  outcomes 

a)  Provide  progress 
status 

Process  and  product  metric  data  have  been  disseminated  according  to  the  agreement, 
engineering  plan,  and  enterprise  policies  and  procedures,  and  to  meet  approved 
requests. 

b)  Provide  planning 
information 

Work  packages  and  appropriate  technical  plans  have  been  disseminated  to  project 
teams  and  other  required  or  approved  recipients. 

c)  Disseminate 
approved  and 
controlled 
requirements 

Acquirer/assigned,  other  stakeholder,  system  technical  and  derived  technical 
requirements,  and  all  changes  to  requirements  are  distributed  in  a  timely  manner  to  all 
stakeholders  to  ensure  that  all  work  is  conducted  in  accordance  with  the  latest 
approved  requirements. 

d)  Provide  information 
for  and  from 
reviews 

The  following  have  been  disseminated,  as  appropriate:  (1)  read-ahead  technical 
review  package  to  technical  review  board  members,  (2)  information  and  items 
necessary  to  demonstrate  that  event-based  criteria  have  been  satisfied  for  initiation  of 
the  review,  (3)  information  packages  and  presentation  materials  at  the  review,  (4) 
minutes  of  the  review  action  items  required  for  closure,  and  final  close-out  approval. 

e)  Make  available 
design  data  and 
schema 

Data  pertinent  for  the  technical  effort  have  been  disseminated  to  project  teams  and 
team  members  to  ensure  information  availability  for  decisions  and  events,  and  to 
other  authorized  recipients  requesting  information. 

f)  Make  available 
lessons  learned 

Lessons  learned  have  been  disseminated  to  other  projects  within  the  enterprise  and  to 
other  teams  within  the  project. 

g)  Report  variances 

Product  and  process  variances  have  been  reported  along  with  (1)  recommended 
actions  to  return  the  product  or  process  metric  to  established  expectations  or 
requirements,  (20  cost  and  schedule  impacts,  and  (3)  effects  on  the  project  if  no 
action  is  taken. 

h)  Disseminate  data 
deliverables 

Data  deliverables  have  been  disseminated  as  required  by  the  agreement,  enterprise 
policies  and  procedures,  the  engineering  plan,  and  other  technical  plans. 

i)  Disseminate 

approved  changes 

Approved  requirements  and  design  changes  and  updated  plans  have  been  distributed 
to  approved  or  required  recipients. 

j)  Disseminate 
directives 

Work  directives  resulting  from  management  decisions  have  been  disseminated  to 
intended  recipients  that  initiate  or  change  work  by  project  teams  or  support 
organizations  within  the  enterprise. 
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Table  C.14  -  Sub-process  14  (Requirements  Definition  Process  -  Acquirer  Requirements) 


Representative  tasks 

Expected  outcomes 

a)  Identify,  collect,  and 
prioritize  acquirer’s 
system  requirements 

User,  customer,  or  assigned  requirements  for  a  system,  or  portion  thereof,  have  been 
identified  and  defined  in  terms  of  needs,  expectations,  capabilities,  and  priorities,  or 
of  assigned  requirements  for  a  system,  or  portion  thereof,  as  expressed  in 
specifications.  Specifically,  the  following  have  been  identified,  as  applicable: 

1)  concept  of  operation; 

2)  what  the  acquirer  wants  the  products  of  the  system  to  accomplish  (functional 
requirements); 

3)  how  well  each  function  must  be  accomplished  (performance  requirements); 

4)  natural  and  induced  environments  in  which  the  system  must  operate  or  be 
used; 

5)  design  constrains  such  as  use  of  non-developmental  or  reusable  items; 

6)  requirements  pertaining  to  the  availability,  electro-magnetic  compatibility, 
health  factors,  human  factors,  interoperability,  maintainability,  reliability, 
safety  and  security; 

7)  measures  of  effectiveness  (MOEs)  that  reflect  overall  expectations  against 
which  satisfaction  will  be  determined;  and 

8)  constraints  pertaining  to  development,  production,  test, 
deployment/installation,  training,  support/maintenance,  and  disposal. 

b)  Ensure 

completeness  and 
consistency  of  the 
set  of  collected 
acquirer 
requirements 

The  collected  user,  customer,  or  assigned  requirements  are  validated.  Resolution  of 
all  conflicts  and  variances  is  completed.  Invoked  the  Requirements  Validation 

Process,  Sub-process  26. 

c)  Record  the  set  of 
acquirer 
requirements 

Validated  set  of  acquirer  requirements  is  captured  in  the  established  information 
database. 

Table  C.15  -  Sub-process  15  (Requirements  Definition  Process  -  Other  Stakeholder  Requirements) 


Representative  tasks 

Expected  outcomes 

a)  Identify  and  collect 
other  stakeholders’ 
end  product 
requirements 

Other  types  of  requirements  that  can  constrain  the  engineering  of  the  system’s  end 
products  are  identified,  collected,  and  defined,  such  as: 

1)  project  plans; 

2)  team  assignments  and  organization; 

3)  automated  tools  availability  and  approval  for  use; 

4)  required  metrics; 

5)  decisions  from  management  or  technical  reviews; 

6)  enterprise  standards,  guides,  policies,  and  procedures; 

7)  enterprise  technologies;  and 

8)  enterprise  physical  and  financial  resources. 

b)  Identify  and  collect 
other  stakeholders’ 
enabling  product 
requirements 

Enabling  product  requirements  associated  with  manufacturing/production,  test, 
deployment/installation,  training,  support,  and  disposal  (including  disposal)  processes 
including  enterprise  capacities  (facilities,  equipment,  tools,  and  staff)  to  accomplish 
these  processes  are  identified,  collected,  and  defined. 
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c)  Identify  and  collect 
other  stakeholders’ 
external  constraints 

Other  end  product  and  development  process  constraints  from  external  sources  are 
identified,  collected,  and  defined,  such  as; 

1)  national  and  international  standards,  laws,  and  regulations  (including 
environmental  protection,  hazardous  material  exclusion  list,  and  waste 
disposal); 

2)  technology  base; 

3)  industry  and  international  standards  and  general  specifications; 

4)  competitor  product  capabilities  and  trends;  and 

5)  interfaces  with  other  existing  or  evolving  systems  and  platforms. 

d)  Ensure 

completeness  and 
consistency  of  the 
set  of  other 
stakeholders’ 
requirements 

The  collected  set  of  other  stakeholder  requirements  is  validated.  Resolution  of  all 
conflicts  and  variances  is  completed.  Invoked  the  Requirements  Validation  Process 
Sub-process  27. 

e)  Record  the  set  of 
other  stakeholder 
requirements 

Validated  set  of  other  stakeholder  requirements  is  captured  in  the  established 
information  database. 

Table  C.16  -  Sub-process  16  (Requirements  Definition  Process  -  System  Technical  Requirements) 


Representative  tasks 

Expected  outcomes 

a)  Establish  required 
transformation  rules, 
priorities,  inputs, 
outputs,  states, 
modes,  and 
configurations 

Transformation  rules,  priorities,  inputs,  outputs,  states,  modes,  and  configurations 
that  will  influence  and  affect  the  other  tasks  for  definition  of  system  technical 
requirements  are  identified  and  defined,  as  appropriate  to  each  system  product. 

b)  Define  operational 
requirements 

The  range  of  anticipated  use  of  the  end  products,  as  identified  in  the  concept  of 
operations  or  specification,  or  for  potential  end  products,  is  defined,  including  for 
each  operational  profile,  the  definition  of: 

1 )  the  utilization  environment  and  factors,  natural  or  induced,  that  can  affect 
end  product  performance; 

2)  the  events  to  which  end  products  must  respond; 

3)  the  physical  and  functional  interfaces  (e.g.,  mechanical,  electrical,  thermal, 
data,  and  procedural)  including  physical  interactions  (e.g.,  form  and  fit), 
system  boundaries  (what  is  controlled  by  the  developer)  and  interactions 
(e.g.,  information  flows  and  behaviors)  of  products  or  environments  within 
developer  control  and  those  systems  or  environments  outside  system 
boundaries; 

4)  what  system  end  products  must  be  able  to  accomplish  (functional 
requirements)  to  satisfy  acquirer  identified  requirements.  Includes  factors 
such  as  producibility,  testability,  transportability,  installability,  operability, 
supportability,  disposability,  reliability,  availability,  maintainability, 
security,  and  safety;  and 

5)  how  often  end  products  will  be  used,  cycle  time  between  use,  and  how  often 
each  product  function  will  be  accomplished. 
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c)  Define  performance 
requirements 

The  following  are  defined;  (1)  the  performance  expectations  for  each  functional 
requirement  (how  well  the  function  must  be  accomplished),  (2)  the  set  of  measure  of 
performance  (MOPs),  made  up  of  the  functional  and  performance  requirements 
combinations,  associated  with  each  MOE,  (3)  the  key  performance  parameters 
(KPPs)  selected  from  the  MOPs  that  will  be  key  indicators  of  end  product  or  system 
performance,  and  if  not  met,  that  will  cause  the  associated  MOE  to  not  be  satisfied 
and  will  put  the  project  in  cost,  schedule,  or  performance  risk,  and  (4)  functional  and 
performance  testability  approach  for  each  requirement  statement. 

d)  Analyze  acquirer 
and  other 
stakeholder 
requirements  to: 

1)  Define  human 

factors  effects 

2)  Establish  capacities 

and  timing 

3)  Define  technology 

constraints 

4)  Define  product 

design  constraints 

5)  Define  enabling 

product 

requirements 

6)  Identify  conflicts 

7)  Determine  Trade-off 

analysis  criteria 

The  following  are  identified  and  defined,  as  applicable: 

1)  the  user  or  operator  roles,  as  applicable,  and  the  human  factor  effects 
(ergonomic  limitations,  work  space,  eye  movement,  access,  cultural 
background,  natural  and  induced  environmental  constraints,  work  tasks,  and 
time  constraints)  associated  with  functional  performance  requirements  on 
potential  users,  operators,  installers,  or  recipients  and  handlers  of  the  system 
end  products 

2)  required  capacities  (e.g.,  memory,  storage,  and  flows)  of  end  products  and 
timing  of  events,  states,  modes,  and  functions  related  to  each  operational 
profile 

3)  any  constraints  or  limitations  from  use  of  existing  technologies  and  the  risks 
associated  with  using  any  unproven  technologies 

4)  any  constraints  that  will  influence  or  affect  end  product  design  (e.g., 
materials,  special  skills,  and  automated  tools),  required  physical 
characteristics  (e.g.,  size,  color,  texture,  weight,  and  buoyancy),  operator 
safety,  system  security,  reuse  requirements,  standardization  of  end  products, 
open  system  architecture,  maintainer  access,  handling  and  storage, 
transportability,  and  other  attributes  of  end  products  or  design  processes  for 
which  trade-offs  cannot  be  made 

5)  technical  requirements  for  enabling  products  associated  with  processes  to 
develop,  produce,  test,  deploy/install,  operate,  support/maintain,  train,  and 
retire/dispose  of  end  products  under  development  or  being  improved 

6)  conflicts  among  the  requirements  set 

7)  the  set  of  risk,  cost,  schedule,  and  performance  criteria  to  be  used  in 
conducting  trade-off  analyses  for  conflict  resolution. 

NOTES 

1  Developers  are  to  ensure  that  residual  risks  from  constraints  are  not 
significant  to  harm  or  otherwise  prevent  the  system  from  performing  its 
functions,  create  unacceptable  costs,  or  price  the  system’s  end  products  out 
of  competitiveness. 

2  Analyses  of  system  requirements  can  necessitate  consideration  of  existing 
or  possible  physical  solutions  to  ensure  feasibility. 

e)  Challenge 
questionable 
requirements 

Acquirer  and  other  stakeholder  requirements  that  are  of  questionable  utility  or  that 
have  an  unacceptable  risk  of  satisfaction  are  identified  and  resolved. 

f)  Resolve  identified 
conflict  of 
requirements 

Any  conflicts  between  combinations  of  functional  requirements,  performance 
requirements,  or  constraints,  as  well  as  within  respective  sets  of  those  requirements, 
are  resolved.  Invoked  the  System  Analysis  Process,  Sub-process  23. 

g)  Prepare  a  set  of 
acceptable  system 
technical 
requirements 

Associated  assumptions  and  technical  requirement  statements  for  the  system  are 
prepared  and  then  validated.  Invoked  the  Requirements  Validation  Process  Sub¬ 
process  25. 

188 


h)  Ensure 

completeness  and 
consistency  of  the 
set  of  system 
technical 
requirements 

System  technical  requirements  are  validated.  Resolution  of  variances  is  completed. 
Invoked  the  Requirements  Validation  Process,  Sub-process  28. 

i)  Record  the  set  of 
system  technical 
requirements 

The  validation  set  of  system  technical  requirements  and  associated  assumptions  is 
captured  in  the  project’s  information  database  and  maintained  and  controlled 
throughout  the  life  of  the  project. 

NOTE  -  Controlled  maintenance  of  the  system  technical  requirements  in  the 
information  database  allows  for  traceability,  supports  validation,  and  is 
essential  for  change  management. 

Table  C.17  -  Sub-process  17  (Solution  Definition  Process  -  Logical  Solution  Representations) 


Representative  tasks 

Expected  outcomes 

a)  Select  and 

implement  one  or 
more  of  the  four 
approaches  below, 
or  the  approach 
designated  by 
enterprise  policies, 
guides,  or  standards: 

1)  Functional  analysis 

2)  Object-oriented 

analysis 

3)  Structured  analysis 

4)  Information 

modeling 

5)  Other  techniques 

An  abstract  definition  of  the  solution  is  provided  in  the  form  of: 

1)  functional  flow,  timelines,  behaviors,  data  and  control  flows,  states  and  modes, 
functional  failure  modes  and  effects. 

2)  objects  encapsulating  a  partition  and  mapping  of  System  Technical  Requirements 
and  characterized  by  services  (behaviors,  functions  and  operations)  provided  and 
by  encapsulated  attributes  (values,  characteristics,  and  data) 

3)  model  data  and  functions  with  algorithms  derived  from  contextual  diagrams  and 
data  flow  diagrams  used  to  decompose  functions  while  explicitly  showing  the 
data  needed  for  each  function 

4)  data  structures  with  their  functions  and  processing  flows  related  to  the  data  and 
associated  with  assigned  system  technical  requirements 

5)  outcomes  from  other  techniques  (dependent  on  the  nature  of  that  particular 
methodology) 

b)  Establish  sets  of 
logical  solution 
representations  by: 

1) Performing  Trade¬ 

off  analyses 

2)  Identifying  and 

defining  interfaces 

3)  Analyzing  behaviors 

4)  Identifying  and 

defining  states 
and  modes 

5)  Identifying  and 

defining  timelines 

6)  Identifying  and 

defining  data  and 
control  flows 

NOTE  -  There  is  no  set  format  or  form  for  the  various  definitions  of  logical 
solutions.  The  format  or  form  selected  is  that  which  best  defines  the 
functional,  behavior,  or  data  flow  or  data  structure,  as  appropriate,  and  that 
will  allow  best  assignment  to  potential  end  products,  manual  operations,  or 
enabling  products  for  generating  physical  solution  representations. 

One  or  more  sets  of  logical  solution  representations  that  are  appropriate  to  the 
engineering  life-cycle  phase  and  the  system  being  engineered  or  reengineered  have 
been  formed  and  defined,  and  include: 

1)  Acceptable  logical  arrangements  and  sequencing,  or  derivative  representations 
(e.g.,  subfunctions,  timelines,  objects,  data  structures,  and  threads)  defined  by 
invoking  the  System  Analysis  Process,  Sub-process  23. 

2)  Interfaces  related  to  logical  arrangements  and  sequencing,  or  derivative 
representations,  to  include,  for  example,  start  and  end  of  states  and  inputs  and 
outputs  defined.  Interface  attributes  identified  and  defined  that  trigger,  for 
example,  a  behavioral  response,  change  of  state  or  mode,  or  data  flow. 

3)  The  responses  (outputs)  of  the  subfunction,  group  of  subfimctions,  objects,  etc., 
to  stimuli  (inputs)  for  each  operational  profile  identified  and  defined,  as 
appropriate.  Executable  threads  identified  and  defined,  as  appropriate,  through 
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7)  Analyzing  failure 
modes  and 
defining  failure 
effects 

the  logical  arrangements  and  sequencing,  or  derivative  representations. 

4)  The  states  and  modes  for  which  subfunctions,  groups  of  subfunctions,  groups, 
objects,  etc.,  exhibit  different  behaviors  are  identified  and  defined. 

5)  Timelines  associated  with  a  sequence  of  functions,  objects,  etc.,  for  each 
operational  profile  are  defined,  as  appropriate.  Ranges  for  execution  time  and 
conditions  that  cause  normal  and  abnormal  performance  are  identified  and 
defined. 

6)  The  following  are  defined,  as  appropriate,  (1)  data  flows  among  subfunctions, 
groups  of  subfunctions,  objects,  etc.,  for  each  operational  profile,  and  (2) 
execution  controls  of  each  subfunction,  and  among  groups  of  subfunctions  or 
objects,  for  each  operational  profile 

7)  The  functional  or  behavioral  consequences  of  any  specific  functional  failure  that 
represent  significant  safety,  security,  human  factor,  performance,  or 
environmental  hazards  are  determined  and  prioritized.  Alternative  actions  to 
resolve  high-priority  failure  consequences  are  determined. 

c)  Assign  system 
technical 
requirements 
(including 
performance 
requirements  and 
constraints 

System  technical  requirements  (including  performance  requirements  of  a  functional 
requirement  and  constraints)  assigned  to  appropriate  subfunctions,  groups  of 
subfunctions,  objects,  data  structures,  etc. 

NOTE  -  There  can  be  unassigned  system  technical  requirements  after  the 
tasks  of  Sub-process  17  are  completed  (see  the  note  in  Sub-process  17  task 

c). 

d)  Identify,  define,  and 
validate  derived 
technical 
requirement 
statements 

Derived  technical  requirement  statements  prepared  that:  (1)  reflect  requirement 
associated  with  defined  logical  solution  representations  from  tasks  a)  and  b),  (2) 
constitute  expansion  of  previously  defined  derived  technical  requirements  into  more 
detailed  lower  level  requirements,  (3)  represent  system  technical  requirement 
statements  (such  as  range)  that  are  not  appropriate  for  logical  solution  representations 
but  through  analysis  can  be  made  more  specific  (such  as  fuel  capacity,  engine 
efficiency,  and  vehicle  resistance),  and  (4)  individually  and  as  a  set,  are  well 
formulated  in  accordance  with  Sub-process  25. 

e)  Ensure 

completeness  and 
consistency  of 
logical  solution 
representations 

Logical  solution  representations  and  assumptions  are  validated.  Resolution  of 
identified  variances  is  completed.  Invoked  the  Validation  Process  Sub-process  29. 

f)  Record  logical 
solution 

representations  and 
derived  technical 
requirements 

The  following  are  captured  in  the  information  database:  (1)  the  data  generated, 
selected  arrangements  and  sequencing,  assignments  of  system  performance 
requirements,  and  constraints,  (2)  the  validated  sets  of  logical  solution 
representations,  (3)  the  derived  technical  requirements,  along  with  source  rationale 
and  assumptions,  and  (4)  any  unassigned  system  technical  requirements  see  the  note 
in  Sub-process  17  task  c). 
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Table  C.18  -  Sub-process  18  (Solution  Definition  Process  -  Physical  Solution  Representations) 


Representative  tasks 

Expected  outcomes 

a)  Analyze  logical 
solution 

representation  sets, 
assigned  system  and 
derived  technical 
requirements 

The  following  are  determined: 

1)  which  logical  solution  set  or  assigned  requirement  provides  a  requirement  for  an 
enabling  product  associated  with  development,  production,  test, 
deployment/installation,  training,  support/maintenance,  or  disposal; 

2)  which  logical  solution  set  or  assigned  requirement  can  best  be  accomplished 
manually  or  by  facilities,  material,  or  data;  and 

3)  which  logical  solution  set  or  assigned  requirement  can  best  be  accomplished  by 
hardware,  software,  or  firmware  products  (new  or  existing). 

Invoke  the  System  Analysis  Process,  Sub-process  22  and  23,  as  necessary. 

b)  Assign 

representations, 
derived  technical 
requirements  and 
unassigned  system 
technical 
requirements  to 
appropriate  physical 
entities 

The  appropriate  sets  of  functions,  groups  of  functions,  objects,  behaviors,  derived 
technical  requirements,  etc.,  are  assigned  to  appropriate  physical  entities  (e.g.,  sensor, 
engine,  power  source,  storage  device,  structural  frame,  communication  device,  and 
computer)  that  will  make  up  a  physical  solution. 

NOTE  -  This  assignment  to  physical  entities  and  generation  of  alternative 
solutions  composed  of  these  entities  is  tightly  coupled  and  iterative. 

c)  Generate  and  evaluate  alternative  physical  solution  representations  by  performing  the  following  tasks: 

NOTE  -  Appropriate  models  (digital,  hardware  or  software,  or  both,  partial  or  complete)  or  prototypes 
are  normally  created  to  help  avert  risk,  identify  critical  product  characteristics  and  enabling  product 
requirements,  identify  control  requirements  for  product  integrity,  perform  sensitivity  analyses  to 
establish  design  margins,  provide  quantitative  performance  assessments,  and  select  preferred  physical 
solution  representation. 

1)  Identify  and 

Define  Physical 
interfaces 

Physical  interfaces  (human,  form,  fit,  function,  data  flow,  and  interoperability)  among 
specific  physical  entities  that  make  up  each  end  product  physical  solution  alternative, 
among  end  products  that  make  up  the  system,  among  end  products  and  enabling 
products,  and  along  with  end  products  and  other  interfacing  systems,  are  identified 
and  defined.  Physical  interfaces  (internal  to  the  system  and  external)  among  specific 
solutions  selected  for  each  physical  entity  that  make  up  the  selected  physical  solution 
are  designed  and  described. 

2)  Identify  and 
Analyze  Critical 
Parameters 

For  each  identified  key  performance  parameter  (TPM),  the  variability  and  the 
sensitivity  of  each  alternative  physical  solution  to  that  variability  are  identified  and 
defined. 

3)  Identify  and  assess  physical  solution  options: 

(a)  Technology 
requirements 

The  technological  needs  necessary  to  make  each  alternative  solution  effective,  the 
risks  associated  with  introduction  of  new  or  advanced  technologies  to  meet 
requirements,  and  alternative  lower-risk  technologies  that  could  be  substituted  for 
unacceptable  higher  risk  technologies  are  identified  and  assessed. 

(b)  Off-the-shelf 
availability 

The  availability  of  off-the-shelf  end  products  (non-developmental  hardware  or 
reusable  software)  are  identified  and  assessed. 

(c)  Competitive 
considerations 

The  effect  of  design  considerations  to  maintain  or  make  a  physical  solution 
representation  alternative  competitive  with  potential  or  existing  competitor  products 
is  identified  and  assessed. 
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(d)  Failure  modes, 
effects,  and 
criticality 

Further  design  efforts  are  identified  that  will  be  needed  to  accommodate  redundancy 
and  to  support  graceful  degradation  when  the  results  of  failure  modes,  effects,  and 
criticality  of  failure  analyses  have  an  unacceptable  or  high  criticality  rating. 

(e)  Performance 
assessment 

The  degree  to  which  the  performance  requirements  are  satisfied  by  each  alternative 
physical  solution  is  identified  and  assessed. 

(f)  Life  cycle 
considerations 

The  degree  to  which  producibility,  testability,  ease  of  deployment,  installability, 
operability,  supportability,  trainability,  and  disposability  are  considered  in  each 
alternative  physical  solution  is  identified  and  assessed.  Enabling  products  needs, 
requirements  and  constraints  for  the  associated  processes  are  identified,  assessed,  and 
defined. 

(g)  Capacity  to 
evolve 

The  capacity  of  each  alternative  physical  solution  to  evolve,  or  be  reengineered, 
incorporate  new  technologies,  enhance  performance,  increase  functionality,  or  other 
cost-effective  or  competitive  improvements,  once  solution  end  products  are  in 
production  or  in  the  marketplace,  are  identified  and  assessed.  Limitations  that  can 
preclude  the  capacity  of  the  system  to  evolve  are  identified  and  documented. 

(h)  Make  vs.  buy 

The  advantages  and  disadvantages  of  making  the  products  of  the  solution  within  the 
enterprise  or  going  to  an  established  supplier  are  identified  and  assessed. 

(i)  Standardization 
considerations 

The  advantages  and  disadvantages  of  using  standardized  end  products,  protocols, 
interfaces,  etc.,  for  the  physical  solution  are  identified  and  assessed. 

(j)  Integration 
concerns 

The  following  are  identified  and  assessed:  (1)  potential  hazards  to  other  systems, 
operators,  or  the  environment;  (2)  built-in  test  and  fault-isolation  test  requirements; 

(3)  ease  of  access,  ready  disassembly,  use  of  common  tools,  part  count  effect, 
advantage  of  modularity,  standardization,  and  less  need  for  cognitive  skills;  and  (4) 
dynamic  or  static  conflicts,  inconsistencies,  and  improper  functionality  of  the 
integrated  products  of  the  solution. 

4)  Perform  system 
analyses 

Which  physical  solution  option  is  best  for  each  alternative  solution  representation, 
based  on  each  option  individually  or  in  sets  (Sub-process  22,  23,  and  24)  is 
determined. 

d)  Identify  and  define 
derived  technical 
requirements 

Derived  technical  requirement  statements  identified  and  defined  that  are:  (1)  the 
consequence  of  design  choices  associated  with  the  above  tasks,  (2)  used  to  form 
alternative  physical  solution  representations,  as  appropriate,  and  (3)  individually  and 
as  a  set  (including  physical  interface  requirements)  well  formulated  (Sub-process  25) 

e)  Select  preferred 
physical  solution 

The  preferred  physical  solution  representation  is  selected,  based  on  the  results  of  an 
evaluation  of  each  physical  solution  representation  (Sub-process  22,  23,  and  24). 

f)  Ensure  selected 
physical  solution 
representation 
consistency 

The  selected  physical  solution  representation  is  determined  to  be  consistent  with 
assigned  logical  solution  representations,  derived  technical  requirements,  and  the 
identified  subset  of  unassigned  system  technical  requirements  [see  the  note  under 
Requirement,  task  17c)] 

g)  Record  the 
outcomes  of  a) 
through  g) 

The  following  are  captured  in  the  information  database:  selected  physical  solution 
representation,  along  with  selection  rationale,  assumptions,  and  outcomes  from  tasks 
a)  through  g). 
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Table  C.19  -  Sub-process  19  (Solution  Definition  Process  -  Specified  Requirements) 


Representative  tasks 

Expected  outcomes 

a)  Fully  characterized 
design  solution 

For  each  specific  physical  entity  of  the  selected  physical  solution:  hardware  drawings 
and  schematics,  software  design  documents,  parts  lists,  interface  descriptions, 
procedural  manuals,  data  or  other  applicable  design  descriptions,  based  on  the 
requirements  assigned  to  the  selected  physical  solution  and  engineering  life-cycle- 
phase  exit  criteria,  are  completed,  as  applicable. 

b)  Ensure  design 

solution  consistency 

The  defined  design  solution  is  verified  as  being  consistent  with  the  selected  physical 
solution  representations  as  described  by  its  encapsulated  requirements  for  the 
assigned  logical  solution  representations,  associated  system  technical  requirements, 
and  derived  technical  requirements.  Invoked  the  Verification  Process  Sub-process 

30. 

c)  Specify 

requirements 

System,  subsystem,  and  interface  specifications  that  describe  the  specified 
requirements  (functional  and  performance  requirements,  and  physical  characteristics) 
are  documented.  Test  requirements  to  ensure  that  end  products  satisfy  their  specified 
requirements  are  determined  and  included  in  the  related  specification,  as  appropriate 
to  the  engineering  life-cycle  phase. 

d)  Record  design 

solution  and  related 

specified 

requirements 

The  design  solution  work  products,  including  the  specified  requirements,  are  captured 
and  recorded  in  the  established  information  database,  along  with  all  trade-off 
analyses,  design  rationale,  assumptions,  and  key  decisions  to  provide  traceability  of 
requirements  up  and  down  the  system  structure. 

e)  Establish  projects 
for  development  of 
enabling  products 

A  project  is  established  to  engineer  the  enabling  products  associated  with  the 
processes  for  development,  production,  test,  deployment/installation,  training, 
support/maintenance,  and  retirement/disposal. 

NOTE  -  The  requirements  for  enabling  products  come  from:  (1)  user  or 
customer  or  assigned  requirements  and  other  stakeholder  requirements  for 
the  system,  and  (2)  derived  technical  requirements  for  end  products  and  their 
subsystems  generated  by  tasks  of  the  Solution  Definition  Process.  Thus, 
initiation  of  enabling  product  development  is  dependent  on  the  completion 
of  the  design  solution  for  the  system  (building  block)  being  engineered  or 
reengineered. 

Table  C.20  -  Sub-process  20  (Implementation  Process) 


Representative  tasks 

Expected  outcomes 

a)  Acquire  products 
(Goods  or  Services) 

Hardware,  software,  firmware  end  products,  or  composites  of  end  products  built  or 
coded  to  their  specified  requirements,  drawings  or  descriptive  documents;  or  other 
needed  physical  entities  for  example,  trained  personnel,  certified  facilities,  special 
techniques  (manual  procedures  or  processes),  manuals)  are  acquired.  Hardware  items 
are:  (1)  purchased  off-the-shelf  from  a  supplier  or  vendor;  (2)  fabricated  in-house;  or 
(3)  from  in-house,  off-the-shelf  supply.  Software  items  are:  (1)  purchased  from  a 
supplier  or  vendor;  (2)  coded  in-house;  or  (3)  reused. 

b)  Validate  acquired 
products 

Acquired  products  are  validated  that  each  acquired  end  product  or  aggregation  of  end 
products  is  in  conformity  with  its  user,  customer,  or  assigned  requirements.  Invoked 
in  the  End  Products  Validation  Process,  Sub-process  33. 

NOTE  -  This  validation  is  accomplished  by  the  supplier  as  per  the 
agreement  or  by  the  acquirer,  with  or  without  supplier  participation.  This 
validation  includes  product  certification  or  acceptance  testing,  as 
appropriate. 
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c)  Assemble/integrate 
validated  end 
products 

End  products  or  aggregations  of  end  products  already  validated  are  physically 
integrated  or  assembled  into  the  required  test  article  or  the  end  product  that  will  be 
verified  and  delivered  to  an  acquirer. 

d)  Verify  integrated 
end  products 

End  products  are  verified  that  each  end  product  of  the  system  under  development 
complies  with  its  specified  requirements.  Invoked  the  System  Verification  Process, 
Sub-process  3 1 :  End  Product  Verification. 

e)  Verify  enabling 
products  for  each 
associated  process 

Enabling  products  for  production,  test,  deployment/installation,  training, 
support/maintenance,  and  retirement/disposal,  as  appropriate,  are  verified  that  they 
will  be  ready  to  perform  the  support  functions  required  by  the  system’s  end  products. 
Associated  processes  are  proofed,  as  applicable.  Invoked  the  System  Verification 
Process,  Sub-process  32:  Enabling  Products  Readiness. 

f)  Validate  the  verified 
end  product 

End  products  are  validated  prior  to  delivery  to  their  acquirer,  if  required  in  the 
agreement,  using  the  End  Products  Validation  Process,  Sub-process  33:  End  Products 
Validation. 

Table  C.21  -  Sub-process  21  (Transition  to  Use  Process) 


Representative  tasks 

Expected  outcomes 

a)  Acquire  and  put  in 
place  enabling 
products 

Appropriate  enabling  products  for  supporting  the  Transition  to  Use  Process  are 
acquired  and  put  in  place. 

b)  Prepare  end 
products  for 
shipping  or  storage 

In  accordance  with  the  agreement:  (1)  packing  materials  and  containers  are  prepared; 
and  (2)  end  products  are  packaged  and  appropriately  labeled  for  either  storage  or 
delivery. 

c)  Store  or  deliver  end 
products 

End  products  awaiting  shipping  are  appropriately  stored  or,  in  accordance  with  the 
agreement,  delivered  to  intended  usage  sites  in  a  condition  suitable  for  application, 
use,  installation,  or  integration  with  other  end  products  or  composites  of  end 
products. 

d)  Prepare  the 

operational  sites 

Sites  where  products  will  be  stored,  installed,  used,  or  maintained,  or  where  services 
will  be  performed,  are  prepared,  as  required  by  the  agreement. 

e)  Installation  of 
products 

End  products  are  installed  at  appropriate  sites,  as  required  by  the  agreement. 

f)  Perform 

commissioning 

Delivered  or  installed  products  are  brought  to  operational  readiness,  with  appropriate 
acceptance  and  certification  tests  completed,  as  required  by  the  agreement. 

g)  Provide  ghosting 

Parallel  operation  of  a  new  and  legacy  end  product  provides  continuing  service  until 
the  new  system  is  fully  on  line  and  accepted  by  the  customer,  as  required  by  the 
agreement. 

h)  Train  users  and 
maintenance 
personnel 

Training  of  users,  operators,  maintainers,  and  other  necessary  personnel  is  completed, 
as  required  by  the  agreement. 

i)  Provide  in-service 
support 

In-service  support  is  provided,  as  required  in  the  agreement. 
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Table  C.22  -  Sub-process  22  (System  Analysis  Process  -  Effectiveness  Analysis) 


Representative  tasks 

Expected  outcomes 

a)  Plan  effectiveness 
analyses 

A  plan  is  prepared  to  include  the  purpose,  objectives,  execution  and  data  collection 
requirements,  schedule  of  tasks,  availability  of  required  resources,  expected 
outcomes,  and  the  general  approach  for  required  effectiveness  analyses. 

b)  Analyze  system  cost 
effectiveness 

For  each  alternative  physical  solution  representation,  as  well  as  for  the  design 
solution,  the  system  cost  effectiveness  is  determined  with  respect  to  the  following 
attributes,  as  applicable:  accuracy,  availability,  capacity,  maintainability,  reliability, 
responsiveness,  operability,  safety,  security,  survivability,  spare  requirements, 
transportability,  vulnerability,  etc. 

c)  Analyze  total 
ownership  cost 

Costs  to  the  enterprise  and  to  the  acquirer  for  alternative  physical  solution 
representations,  for  alternative  trade-off  analysis  options,  or  for  proposed  changes, 
and  the  known  uncertainties  (risks)  in  these  costs  are  determined. 

NOTE  -  The  following  costs  are  typically  included  in  a  total  ownership  cost 
analysis:  development,  production,  test,  deployment/installation,  training, 
operations,  support/maintenance,  and  retirement/disposal. 

d)  Analyze 

environmental 

impacts 

Applicable  federal,  state,  municipal,  and  international  environmental  statutes  and 
applicable  hazardous  material  lists  affecting  the  project  and  endurance  of  compliance 
by  each  physical  solution  are  determined;  the  effect  on  and  by  each  end  product  and 
enabling  product  on  the  infrastructure,  land  and  ocean,  atmosphere,  water  sources, 
and  animal,  plant  and  human  life,  as  applicable,  has  been  determined,  from  an 
enterprise-based  life  cycle  perspective. 

e)  Analyze  system 
effectiveness 

For  each  operational  profile,  each  alternative  physical  solution  representation  and  the 
design  solution  are  assessed  by  analytic  confirmation  to  satisfy  appropriate 
requirements. 

f)  Record  outcomes  of 
effectiveness 
analyses 

Effectiveness  analysis  outcomes,  as  well  as  the  details  of  the  analyses  performed, 
including  rationale,  assumptions,  and  lessons  learned,  are  captured  and  recorded  in 
the  established  information  database. 

Table  C.23  -  Sub-process  23  (System  Analysis  Process  -  Trade-off  Analysis) 


Representative  tasks 

Expected  outcomes 

a)  Plan  Trade-off 
analysis 

A  plan  is  prepared  to  include: 

1)  the  availability  of  required  resources,  level  of  importance,  execution  and  data 
collection  requirements,  expected  outcomes,  objectives,  schedule  of  tasks,  and 
the  type. 

NOTES  -  The  types  of  trade-off  analyses  typically  performed  include: 

1  Formal  -  formally  conducted,  with  results  reviewed  at  technical  reviews. 
Specific  formal  trade-off  analyses  are  normally  identified  in  an  agreement. 

2  Informal  -  follows  the  same  methodology  of  a  formal  trade-off  analysis  but 
requires  less  documentation  and  is  of  less  importance  to  the  acquirer. 

3  Judgmental  -  selection  of  a  recommended  option,  based  on  judgment  of  the 
analyst  or  designer  after  a  less  rigorous  assessment. 

2)  Selection  criteria  that  characterize  what  makes  a  specific  option  desirable  or 
undesirable,  such  as  (1)  cost,  schedule,  performance,  and  risk;  (2)  life-cycle 
concerns;  (3)  -ility  concerns  (e.g.,  producibility,  testability,  maintainability, 
supportability,  disposability);  (4)  size,  weight,  and  power  consumption  for  the 
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type  of  Trade-off  analysis  selected;  and  (5)  effectiveness  analysis  outcomes. 

3)  weighting  factors  for  each  criteria  on  that  will  help  distinguish  its  degree  of 
importance  for  the  defined  trade-off  analysis. 

4)  applicable  models  (representative  or  simulation)  that  will  support  conduct  of  the 
trade-off  analysis,  as  well  as  determination  that  the  model  selected  is  valid  for 
the  trade-off  analysis  to  be  performed. 

5)  list  of  viable  optional  solutions  or  courses  of  action  to  be  evaluated. 

b)  Perform  Trade-off 
analysis 

Trade-off  analyses  are  completed  according  to  the  plan,  with  determination  of: 

1)  quantitative  basis  for  evaluating  the  trade-off  analysis  options  from  appropriate 
effectiveness  analysis  tasks  (Sub-process  22); 

2)  quantitative  assessment  of  the  risk  level  associated  with  each  option  from 
appropriate  risk  analysis  tasks  (Sub-process  24);  and 

3)  collection  of  data  pertaining  to  each  option  evaluated  and  analysis  of  the  data  to 
determine  the  effect  of  each  option  on  the  system  or  project  if  implemented. 
Determination  that  the  methodologies  and  data  collection  were  sufficient  to 
support  a  fair  and  complete  evaluation. 

4)  Identification  and  definition  of  the  recommended  option  based  on  the 
comparison  of  each  option  and  its  effects  against  the  established  success  criteria. 

5)  Presentation  of  the  recommendations  to  the  appropriate  decision  maker,  as 
applicable. 

c)  Record  outcomes  of 
Trade-off  analysis 

Recommendations  and  the  selection,  as  well  as  the  details  of  the  trade-off  analysis 
performed,  including  rationale,  assumptions,  and  lessons  learned,  are  captured  and 
recorded  in  the  established  project  information  database. 

Table  C.24  -  Sub-process  24  (System  Analysis  Process  -  Risk  Analysis) 


Representative  tasks 

Expected  outcomes 

a)  Identify  risks 

Technical  risks,  and  resulting  project  risks,  are  identified,  based  on  exposure  to  the 
probability  of  an  undesirable  consequence  and  the  effect  of  that  consequence  for  each 
trade-off  analysis  option  or  each  physical  solution  representation  option. 

Considerations  include  how  expectations  from  a  decision  or  design  selection  are 
affected  by  (1)  commitments  resulting  from  a  choice,  (2)  validity  of  assumptions,  (3) 
capabilities  to  implement  and  control,  and  (4)  other  organizational  or  technical 
constraints  such  as  resources  and  time. 

b)  Characterize  risks 

Risk  causes,  possible  effects  or  consequences,  likelihood  of  occurrence,  options  for 
dealing  with  identified  risks,  how  long  options  are  available,  and  coupling  among 
identified  risks  are  determined. 

c)  Prioritize  risks 

Risks  that  would  likely  cause  harm,  would  have  the  greatest  effect,  and  would  need 
immediate  attention  are  prioritized. 

d)  Evaluate  ways  to 
avert  risks 

The  cost,  schedule,  and  performance  effects  on  the  project  are  determined  from 
evaluation  of  options  or  courses  of  action  that  would  (1)  eliminate  a  specific  risk 
possibility;  (2)  implement  acts  to  reduce  a  risk’s  probability  or  effect;  (3)  transfer  the 
risk  (get  someone  else  to  assume  the  risk,  e.g.,  a  warranty);  or  (4)  provide  a 
contingency  to  address  the  consequences,  if  the  risk  occurs,  including  identity  or 
appropriate  and  timely  triggers  for  taking  action  (will  they  give  sufficient  time  to 
act?)  such  as  a  metrics  or  events  monitor. 
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e)  Define  and 

implement  a  plan  or 
approach  for 
averting  each 
significant  risk 

The  significant  risks  to  the  project  are  identified  and  adequate  risk  aversion 
approaches  are  defined.  Triggers  are  defined  that  will  provide  a  signal  when  it  is 
appropriate  to  implement  aversion  action.  Implemented  planned  actions  or 
approaches  to  avert  risk. 

f)  Capture  and 

communicate  risk 
analysis  outcomes 

The  effects  of  the  risk  analysis,  as  well  as  the  details  of  the  risk  analysis  performed, 
including  assumptions,  are  captured  and  recorded  in  the  established  project 
information  database.  Risks  effects  have  been  reported  or  used,  as  appropriate. 

Table  C.25  -  Sub-process  25  (Requirements  Validation  Process  -  Requirement  Statements  Validation) 

Representative  tasks  Expected  outcomes 

a)  Analyze  and  ensure  each  technical  requirement  statement  is  stated  with: 

1)  ability  to  preserve  competitiveness  -  permits  preservation  of  a  competitive  stance  and  is  only  as 
constraining  on  competitive  stance  as  is  justified  by  benefits  delivered  by  requirement. 

2)  clarity  -  requirement  statement  is  readily  understandable  without  analysis  of  meaning  of  words  or  terms 
used. 

3)  correctness  -  requirement  statement  does  not  contain  an  error  of  fact. 

4)  feasibility  -  requirement  can  be  satisfied  within  (1)  natural  physical  constraints,  (2)  state  of  the  art  as  it 
applies  to  the  project,  and  (3)  all  other  absolute  constraints  applying  to  the  project. 

5)  focus  -  requirement  is  expressed  in  terms  of  ‘what’  and  ‘why’,  or  form,  fit  and  function,  not  in  terms  of 
how  to  develop  the  products  or  the  materials  to  be  used  -  detailed  requirements  that  are  required  to  guide 
detailed  design  of  a  product  are  an  exception  to  this. 

6)  implementability  -  requirement  statement  contains  information  necessary  to  enable  requirement  to  be 
implemented. 

7)  modifiability  -  necessary  changes  to  a  requirement  can  be  made  completely  and  consistently 

8)  removal  of  ambiguity  -  allows  only  on  interpretation  for  meaning  of  the  requirement,  e.g.,  not  defined 
by  words  or  terms  such  as  ‘excessive,’  ‘sufficient,’  and  ‘resistant’  that  cannot  be  measured. 

9)  singularity  -  requirement  statement  cannot  be  sensibly  expressed  as  two  or  more  requirements  having 
different  agents,  actions,  objects,  or  instruments. 

10)  testability  -  existence  of  finite  and  objective  process  with  which  to  verify  that  the  requirement  has  been 
satisfied. 

11)  verifiability  -  can  be  verified  at  the  level  of  system  structure  at  which  it  is  stated. _ 

b)  Analyze  and  ensure  technical  requirements  statements  in  pairs  and  as  a  set  are  stated  with: 

1)  absence  of  redundancy  -  each  requirement  is  specified  only  once. 

2)  connectivity  -  all  terms  within  a  requirement  are  adequately  linked  to  other  requirements  and  to  work 
and  term  definitions,  so  that  individual  requirements  relate  properly  to  other  requirements  as  a  set. 

3)  removal  of  conflicts  -  requirement  is  not  in  conflict  with  other  requirements  or  within  itself. _ 
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Table  C.26  -  Sub-process  26  (Requirements  Validation  Process  -  Acquirer  Requirements  Validation) 


Representative  tasks 

Expected  outcomes 

a)  Select  methods  and 
define  procedures 

The  methods  and  procedures  for  validating  the  set  of  defined  acquirer  requirements 
are  selected  and  defined,  consistent  with  the  level  of  system  structure,  enterprise- 
based  life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

b)  Establish  downward 
traceability 

The  downward  traceability  of  stated,  documented,  or  otherwise  determined,  acquirer 
needs  and  expectations  to  the  set  of  defined  acquirer  requirements  is  determined. 

c)  Establish  upward 
traceability 

The  upward  traceability  of  the  individual  acquirer  requirements,  from  the  set  of 
defined  acquirer  requirements,  to  stated,  documented,  or  otherwise  captured,  acquirer 
needs  and  expectations  is  determined. 

d)  Identify  and  resolve 
variances 

Identified  voids,  variances,  and  conflicts  have  been  resolved.  When  the  set  of  defined 
acquirer  requirements  is  not  upward-traceable  to  acquirer  needs  and  expectations, 
whether  non-sourced  (orphaned)  requirements  or  constraints  were  introduced  and 
whether  they  are  desired  by  the  acquirer,  have  been  determined,  and  appropriate 
action  has  been  taken.  When  acquirer  needs  and  expectations  are  not  reflected  in  the 
set  of  defined  acquirer  requirements,  the  omitted  needs  and  expectations  are  added  to 
the  set  of  defined  acquirer  requirements,  as  appropriate. 

e)  Record  validation 
results 

Validation  procedures,  outcomes,  assumptions,  corrective  actions,  lessons  learned, 
etc.,  are  captured  and  recorded  in  the  established  information  database. 

Table  C.27  -  Sub-process  27  (Requirements  Validation  Process  -  Other  Stakeholder  Requirements 

Validation) 


Representative  tasks 

Expected  outcomes 

a)  Select  methods  and 
define  procedures 

The  methods  and  procedures  for  validating  the  set  of  defined  other  stakeholder 
requirements  are  selected  and  defined  and  are  consistent  with  the  level  of  system 
structure,  enterprise-based  life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

b)  Establish  downward 
traceability 

The  downward  traceability  of  stated,  documented,  or  otherwise  determined,  other 
stakeholder  needs  and  expectations  to  the  set  of  defined  other  stakeholder 
requirements  is  established. 

c)  Establish  upward 
traceability 

The  upward  traceability  of  the  individual  other  stakeholder  requirements,  from  the  set 
of  defined  other  stakeholder  requirements,  to  stated,  documented,  or  otherwise 
captured,  other  stakeholder  needs  and  expectations  is  established. 

d)  Identify  and  resolve 
variances 

Identified  voids,  variances,  and  conflicts  are  resolved.  When  the  set  of  defined  other 
stakeholder  requirements  was  not  upward-traceable  to  other  stakeholder  needs  and 
expectations,  whether  non-sourced  (orphaned)  requirements  or  constraints  were 
introduced,  has  been  determined,  and  appropriate  actions  were  taken  to  eliminate 
non-sourced  requirements.  When  other  stakeholder  needs  and  expectations  were  not 
reflected  in  the  set  of  defined  other  stakeholder  requirements,  omitted  needs  and 
expectations  were  added  to  the  set  of  defined  other  stakeholder  requirements,  as 
appropriate. 

e)  Record  validation 
results 

Validation  procedures,  outcomes,  assumptions,  corrective  actions,  lessons  learned, 
etc.,  are  captured  and  recorded  in  the  established  information  database. 
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Table  C.28  -  Sub-process  28  (Requirements  Validation  Process  -  System  Technical  Requirements  Validation) 


Representative  tasks 

Expected  outcomes 

a)  Select  methods  and 
define  procedures 

The  methods  and  procedures  for  validating  the  set  of  defined  system  technical 
requirements  are  selected  and  defined  and  are  consistent  with  the  level  of  system 
structure,  enterprise-based  life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

b)  Establish  downward 
traceability 

The  downward  traceability  of  the  validated  sets  of  stakeholder  (acquirer  and  other 
stakeholder)  requirements  to  the  set  of  defined  system  technical  requirements  is 
determined. 

c)  Establish  upward 
traceability 

The  upward  traceability  of  the  individual  system  technical  requirements,  from  the  set 
of  defined  system  technical  requirements,  to  the  validated  sets  of  stakeholder 
requirements  is  determined. 

d)  Analyze 
assumptions 

Assumptions  regarding  consistency  of  the  system  technical  requirements  with  the 
system  being  engineered  are  determined. 

e)  Analyze  other 
system  technical 
requirements 

Other  system  technical  requirements  derived  as  essential  to  design  and  subsequent 
life  cycle-phases  are  consistent  with  the  system  being  engineered  and  other  system 
technical  requirements  are  determined. 

f)  Identify  and  resolve 
variances 

Identified  voids,  variances,  and  conflicts  are  resolved.  When  the  set  of  defined 
system  technical  requirements  was  not  upward-traceable  to  validated  sets  of 
stakeholder  requirements,  whether  non-sources  (orphaned)  requirements  or 
constraints  were  introduced  was  determined,  and  appropriate  actions  to  eliminate 
non-sourced  requirements  or  revised  the  appropriate  set  of  stakeholder  requirements 
were  taken.  When  validated  stakeholder  requirements  were  not  reflected  in  the  set  of 
defined  system  technical  requirements,  omitted  requirements  were  added  to  the  set  of 
defined  system  technical  requirements  or  determine  the  need  for  the  requirement,  as 
appropriate. 

g)  Perform  revalidation 

When  a  change  is  needed  to  one  of  the  validated  sets  of  stakeholder  requirements,  the 
appropriate  tasks  of  acquirer  or  other  stakeholder  requirements  definition  from  the 
Requirements  Definition  Process  were  accomplished  and  the  set  was  revalidated. 

When  the  set  of  system  technical  requirements  must  be  changed,  the  appropriate  tasks 
of  system  technical  requirements  definition  from  the  Requirements  Definition 

Process  were  reaccomplished  and  the  set  was  revalidated. 

h)  Record  validation 
results 

Validation  procedures,  outcomes,  assumptions,  corrective  actions,  lessons  learned, 
etc.,  are  captured  and  recorded  in  the  established  information  database. 

Table  C.29  -  Sub-process  29  (Requirements  Validation  Process  -  Logical  Solution  Representations 

Validation) 


Representative  tasks 

Expected  outcomes 

a)  Select  methods  and 
define  procedures 

The  methods  and  procedures  for  validating  the  defined  sets  of  logical  solution 
representations  and  derived  technical  requirements  are  selected  and  defined  and  are 
consistent  with  the  level  of  system  structure,  enterprise-based  life-cycle  phase,  and 
Validation  Plan,  as  appropriate. 

b)  Establish  downward 
traceability 

The  downward  traceability  of  the  validated  set  of  system  technical  requirements  to 
each  set  of  logical  solution  representations  and  the  derived  technical  requirements  is 
determined. 

c)  Establish  upward 
traceability 

The  upward  traceability  of  individual  logical  solution  representations  from  a  set  of 
logical  solution  representations  and  the  derived  technical  requirements  to  the 
validated  set  of  system  technical  requirements  is  determined. 
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d)  Analyze 
assumptions 

Assumptions  made  while  defining  the  sets  of  logical  solution  representations  to 
ensure  that  they  are  consistent  with  the  system  technical  requirements  and  the  system 
being  engineered  are  assessed  and  considered  valid. 

e)  Identify  and  resolve 
variances 

Identified  voids,  variances,  and  conflicts  are  resolved.  When  validated  system 
technical  requirements  are  not  reflected  in  a  set  of  logical  solution  representations, 
omitted  requirements  are  added  to  the  set  of  logical  solution  representations.  The 
need  for  added  requirements  is  confirmed,  and  it  is  determined  whether  these 
requirements  were  to  be  assigned  directly  to  physical  solutions.  When  a  set  of  logical 
solution  representations  is  not  upward  traceable  to  the  validated  set  of  system 
technical  requirements,  it  is  determined  whether  non-sourced  (orphaned) 
requirements  and  constraints  have  been  introduced.  Appropriate  actions  are  taken 
whether  to  eliminate  non-sourced  requirements,  to  establish  derived  requirements,  or 
to  revise  the  set  of  system  technical  requirements. 

f)  Perform  revalidation 

When  a  change  is  needed  to  the  validated  set  of  system  technical  requirements,  the 
appropriate  tasks  from  the  Requirements  Definition  Process  are  reaccomplished  and 
the  set  is  revalidated.  When  one  or  more  sets  of  logical  solution  representations  has 
to  be  changed,  the  appropriate  tasks  for  definition  of  logical  solution  representations 
from  the  Solution  Definition  Process  are  reaccomplished  and  the  set  was  revalidated. 

g)  Record  validation 
results 

Validation  procedures,  outcomes,  assumptions,  corrective  actions,  lessons  learned, 
etc.,  are  captured  and  recorded  in  the  established  information  database. 

Table  C.30  -  Sub-process  30  (System  Verification  Process  -  Design  Solution  Verification) 


Representative  tasks 

Expected  outcomes 

a)  Plan  the  design 

solution  verification 
in  accordance  with 
the  Verification 

Plan,  the  agreement, 
and  the  applicable 
enterprise-based  life 
cycle-phase,  and 
level  in  the  system 
structure 

1)  The  appropriate  method  needed  to  verify  the  system’s  fully  characterized  design 
solution  is  identified  and  defined 

NOTE  -  Design  solution  verification  methods  include:  inspection  (for 
example,  inspection  of  drawings),  analysis  (for  example,  using  simulation  or 
virtual  reality  prototype),  demonstration  (for  example,  using  mockups  or 
physical  models),  or  test  (for  example,  by  testing  physical  prototypes, 
breadboards,  or  brassboards). 

2)  Verification  procedures  are  defined,  based  on  (1)  procedures  for  each  method 
selected,  (2)  purpose  and  objective  of  each  procedure  (3)  pre-test  and  post-test 
actions,  and  (4)  criteria  for  determining  the  success  or  failure  of  the  procedure 

3)  The  verification  environment  (for  example,  facilities,  equipment,  tools, 
simulations,  measuring  devices,  personnel,  and  climatic  conditions)  in  which  the 
verification  methods  and  procedures  will  be  implemented  is  established  and 
checked-out  for  adequacy,  completeness,  readiness,  and  integration. 

b)  Perform  the  planned 
design  solution 
verification  using 
selected  methods 
and  procedures 
within  the 
established 
verification 
environment 

Verification  outcomes  to  show  completion  of  verification  objectives  and  to  determine 
untraceable  requirements  and  constraints,  voids,  conflicts,  variations  and  anomalies 
are  collected  and  evaluated.  Specifically,  it  was  shown  that: 

1)  the  system  design  solution  descriptions  and  interfaces  (internal  or  external)  are 
upward-traceable  to  requirements  of  the  selected  physical  solution 
representation; 

2)  source  requirements  are  downward-traceable  to  the  system  design  solution 
descriptions; 

3)  the  design  solution  satisfied  the  functional  and  performance  requirements  of  the 
identified  subset  of  unassigned  system  technical  (see  note  under  Sub-process  17 
c)  and  the  set  of  derived  technical  requirements; 
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4)  intended  functions  are  correctly  implemented; 

5)  constraints,  including  interfaces,  are  satisfied. 

When  defined  variances  were  not  downward-traceable  from  source  documents, 
appropriate  tasks  of  the  Requirements  Definition  and  Solution  Definition  Processes 
were  repeated  to  correct  the  omissions.  When  defined  variances  showed 
inconsistencies  with  source  requirements  (not  upward-traceable),  the  following  were 
determined:  why  new  requirements  were  introduced,  and  if  they  were  to  be  assigned 
as  derived  technical  requirements,  were  to  be  removed  from  the  design  solution 
definition,  or  had  to  be  reflected  in  the  set  of  logical  solution  representations  or  set  of 
system  technical  requirement.  The  necessary  tasks  of  the  Requirements  Definition 
and  Solution  Definition  Processes  were  reaccomplished  as  required  for  corrections 
and  reverifications. 

c)  Perform 

reverification 

When  test  outcome  variations  and  anomalies  were  traced  to  poor  verification  conduct 
or  to  inadequate  verification  environment,  verifications  are  repeated  to  obtain  valid 
outcomes. 

d)  Record  verification 
results 

The  verification  procedure,  together  with  the  outcomes  achieved,  variations, 
corrective  actions  taken,  rationale  justifying  the  design  solution,  trade-off  analyses 
and  effectiveness  analyses  completed  with  resulting  key  decisions,  verified  design 
solution  definition,  lessons  learned,  etc.,  are  recorded  in  the  project  information 
database  according  to  the  verification  plan  and  test  procedure  requirements. 

NOTE  -  The  verified  design  solution  and  its  related  specified  requirements 
are  placed  under  configuration  management  control. 

Table  C.31  -  Sub-process  31  (System  Verification  Process  -  End  Product  Verification) 


Representative  tasks 

Expected  outcomes 

a)  Plan  the  end  product 
verification  in 
accordance  with  the 
Verification  Plan, 
the  agreement,  and 
the  applicable 
enterprise-based  life 
cycle-phase,  and 
level  in  the  system 
structure 

1)  The  appropriate  methods  needed  to  verify  the  system’s  end  products  against  their 
specified  requirements  are  selected  and  defined. 

NOTE  -  Design  solution  verification  methods  include:  inspection  (for  example, 
inspection  of  drawings),  analysis  (for  example,  using  simulation  or  virtual  reality 
prototype),  demonstration  (for  example,  using  mockups  or  physical  models),  or 
test  (for  example,  by  testing  physical  prototypes,  breadboards,  or  brassboards). 

2)  Verification  procedures  are  established  and  based  on  (1)  procedures  for  each 
method  selected,  (2)  purpose  and  objective  of  each  procedure,  (3)  pre-test  and 
post-test  actions,  and  (4)  criteria  for  determining  the  success  or  failure  of  the 
procedure. 

3)  The  verification  environment  (for  example,  facilities,  equipment,  tools, 
simulations,  measuring  devices,  trained  personnel,  special  techniques,  and 
climatic  conditions)  in  which  the  verification  methods  and  procedures  will  be 
implemented  is  established  and  checked  out  for  adequacy,  completeness, 
readiness,  and  integration. 

4)  Test  articles  are  on  hand,  assembled,  and  integrated  with  the  verification 
environment  according  to  the  verification  plans  and  schedules,  and  appropriate 
sets  of  specified  requirements  are  available. 

b)  Perform  the  planned 
end  product 
verification  using 
selected  methods 
and  procedures 

Verification  outcomes  are  collected  and  evaluated  to  show  completion  of  verification 
objectives  and  used  to  determine 

1)  variations  and  anomalies,  and  out-of-compliance  conditions; 

2)  data  quality,  integrity,  correctness,  consistency  ,  and  validity; 

3)  whether  fabricated,  integrated,  or  purchased  end  products  (including  end 
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within  the 
established 
verification 
environment 

products,  composites  of  end  products,  or  software  or  firmware  builds) 
comply  with  their  respective  specified  requirements; 

4)  that  end  product  test  articles  were  appropriately  integrated  with  the  test 
environment  and  each  requirement  was  properly  tested  for;  and 

5)  that  system  end  products  function  together  and  with  interfacing  products 
throughout  their  performance  envelope. 

For  variations  and  anomalies  not  caused  by  poor  test  conduct,  or  conditions, 
appropriate  tasks  of  the  processes  in  this  Guide,  including  replanning,  changing 
requirements,  redefining  requirements,  and  the  design  solution,  and  verification,  are 
accomplished  to  resolve  discrepancies. 

c)  Perform 

reverification 

When  test  outcome  variations  and  anomalies  were  traced  to  poor  verification  conduct 
or  to  inadequate  verification  environment,  end  product  verification  is  reaccomplished. 

d)  Record  verification 
results 

The  verification  methods  and  procedures,  together  with  the  outcomes  achieved, 
variations  and  anomalies,  corrective  actions  taken,  rationale  justifying  corrections, 
trade-off  analyses,  and  effectiveness  analyses  completed  with  resulting  key  decisions, 
lessons  learned,  etc.,  are  recorded  in  the  project  information  database  according  to  the 
verification  plan  and  test  procedure  requirements.  Recorded  test  result  data  includes 
the  following: 

1)  The  version  of  the  set  of  specified  requirements  (specifications)  used. 

2)  The  version  of  the  end  product  tested. 

3)  The  version  or  reference  standard  for  tools  and  equipment  used,  together 
with  applicable  calibration  data. 

4)  The  results  of  each  test  including  pass  or  fail  declarations. 

5)  The  discrepancy  between  expected  and  actual  results. 

6)  A  statement  of  success  or  failure  of  the  testing  process,  including  its  relation 
to  the  verification  process. 

Deliver  or  disposition  of  verified  compliance  articles  and  compliance  data  is 
completed  in  accordance  with  the  acquirer- supplier  agreement,  verification  plan 
instructions,  or  project  directives  or  procedures. 

Table  C.32  -  Sub-process  32  (System  Verification  Process  -  Enabling  Product  Verification) 


Representative  tasks 

Expected  outcomes 

a)  Plan  the  enabling 
product  readiness 
determination  in 
accordance  with  the 
agreement,  and  the 
applicable 
enterprise-based  life 
cycle-phase,  and 
level  in  the  system 
structure 

1)  The  appropriate  methods  needed  to  determine  enabling  product  readiness  and 
maturity  of  development,  based  on  the  applicable  enterprise-based  life-cycle 
phase  and  level  in  the  system  structure,  the  purpose  and  objective  of  each  method 
selected,  the  appropriate  plan,  and  the  acquirer- supplier  agreement,  are  selected 
and  defined. 

2)  Procedures  based  on  (1)  each  method  selected,  (2)  purpose  and  objective  of  each 
method,  (3)  pre-test  and  post-test  actions,  and  (4)  criteria  for  determining  the 
success  or  failure  of  the  method  are  established. 

3)  The  environments  (for  example,  facilities,  equipment,  tools,  simulations, 
measuring  devices,  trained  personnel,  special  techniques,  and  climatic 
conditions)  in  which  the  methods  and  procedures  will  be  implemented  is 
established  and  checked  out  for  adequacy,  completeness,  readiness,  and 
integration. 

4)  Required  information  regarding  the  status  and  maturity  of  enabling  product 
development  or  requirements  definition  is  on  hand.  Non-developmental  enabling 
products  are  on  hand  and  integrated  appropriately. 

b)  Perform  the  planned 
enabling  product 
readiness 

Outcomes  are  collected  and  evaluated,  and  any  enabling  product  readiness  anomalies, 
variations,  or  out-of-compliance  conditions  (such  as  lack  of  requirements  for  manuals 
or  training  equipment  or  disposal  of  hazardous  materials)  are  discovered. 
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determination,  using 
selected  methods 
and  procedures 

The  following  have  been  determined 

1)  whether  development  for  required  enabling  products  is  progressing  satisfactorily 
or  will  be  ready  to  perform  its  life  cycle  function  when  needed  or  if  there  are  out- 
of-compliance  conditions. 

2)  that  plans  and  selected  methods,  procedures,  and  tools  for  each  associated 
process  can  accomplish  their  intended  purpose 

3)  whether  the  development  is  on  schedule  and  that  the  schedule  meets  critical  end 
product  needs 

4)  the  interfaces  between  planned  enabling  products  and  their  intended  end  products 
have  no  potential  conflicts  in  implementation  concepts,  intended  functions,  or 
interdependencies 

5)  that  enabling  products  meet  the  requirements  of  the  end  products  or  composites 
of  end  products  they  are  intended  to  support. 

For  variations  and  anomalies  not  caused  by  poor  readiness  assessments,  appropriate 
tasks  of  the  processes  in  this  Guide,  include  replanning,  changing  requirements, 
redefining  requirements  and  the  design  solution,  and  readiness  determination,  are 
accomplished  to  resolve  discrepancies. 

c)  Reaccomplish 
readiness 
determination 

For  discrepancies  caused  by  poor  readiness  assessment,  the  appropriate  tasks  of 
enabling  product  readiness  determination  are  reaccomplished. 

d)  Record  readiness 
determination 
results 

Enabling  product  readiness  determination  outcomes  are  recorded  in  the  information 
database. 

Table  C.33  -  Sub-process  33  (End  Products  Validation  Process) 


Representative  tasks 

Expected  outcomes 

a)  Determine 
validation  exit 
criteria 

The  type  of  validation  required  and  the  requirements  to  be  used  are  determined.  The 
types  include:  (1)  validation  against  acquirer  requirements  in  the  anticipated  usage 
environment,  with  test  conditions  that  span  the  expected  range  of  actual  operating 
conditions,  to  the  extent  practical,  and  in  conjunction  with  stakeholders,  as 
appropriate;  (2)  certification  tests  against  established  certification  requirements;  (3) 
acceptance  tests  using  operational  processes  and  personnel  in  operational 
environments;  or  (4)  as  specified  in  the  agreement. 

NOTES 

1  Validation  tests  are  conducted  during  the  Test  and  Evaluation  Phase  of  the 
engineering  life  cycle,  after  the  end  products  have  been  verified  against  specified 
requirements,  from  the  lowest  level  of  the  system  structure  upward  to  the  end 
products  that  will  be  delivered  to  the  marketplace  to  satisfy  validated  acquirer 
requirements. 

2  Validations  of  Types  1  through  3  are  satisfied  with  the  same  tests,  when 
appropriate. 

3  Validation  can  be  for  a  single  end  product  or  an  aggregation  of  end  products 
for  the  same  building  block. 

b)  Acquire  appropriate 
test  article 

The  test  article,  or  test  articles,  used  for  the  validation  is  determined  to  be 
appropriate  to  the  enterprise-based  life-cycle  phase  and  the  level  of  system  structure. 

NOTE  -  End  Products  Validation  consists  of  one  or  more  tests  using  a  version  of 
the  product  (or  products)  as  nearly  like  the  final  version  as  is  practical  and 
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necessary,  taking  into  account  the  enterprise-based  life-cycle  phase  and  the 
nature  of  the  product.  If  the  nature  of  either  product,  its  operating  conditions,  or 
the  enterprise-based  life-cycle  phase  of  development  precludes  use  of  actual 
products  or  prototypes,  then  breadboards,  brassboards,  hardware-in-the-loop 
simulations,  virtual-reality  simulations,  or  other  models  and  simulations  are 
applicable  for  End  Products  Validation. 

c)  Conduct  validation 

1)  Validation  is  completed  in  accordance  with  the  Validation  Plan,  as  required  in 
the  agreement. 

2)  Validation  outcomes  are  compiled,  analyzed,  and  compared  to  the  validation 
exit  criteria;  variations  and  anomalies  have  been  identified;  and  corrective 
actions  are  defined. 

3)  When  outcome  variances  from  exit  criteria  were  not  caused  by  improper  test 
conditions,  by  improper  performance  of  validation  procedures,  or  by  improper 
data  collection:  Replanning,  redefinition  of  the  design  solution,  and  the 
Implementation  Process,  as  appropriate,  are  reaccomplished. 

NOTE  -  Care  is  to  be  taken  to  ensure  that  the  requirements  derived  to  remove 
variances  do  note  conflict  with  acquirer  or  other  stakeholder  requirements,  or 
other  validated  technical  requirements  without  coordinating  such  change  with  the 
appropriate  stakeholders. 

d)  Perform 
revalidation 

If  variances  were  caused  by  poor  test  conduct,  retesting,  using  improved  or  correct 
test  equipment  and  procedures,  is  performed. 

e)  Record  validation 
results 

Validation  procedures,  compliance  data,  outcomes,  assumptions,  corrective  actions, 
lessons  learned,  etc.,  are  recorded  in  the  established  project  information  database. 
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Appendix  D  -  Planning  Documents  (informative) 


This  Appendix  provides  an  informative  list  of  typical  documents  and  their  contents  taken  from  various 
commercial  and  non-commercial  domains.  Selection  and  use  of  these  documents  depends  on  agreement 
requirement  and  the  nature  and  scope  of  the  project. 

D.1  Source  Documents 

1 .  During  early  phases  of  the  enterprise-based  life  cycle,  system  concepts  are  often  vague  and 
unstructured.  Typical  concept  source  documents  include: 

a)  Concept  Specification.  This  includes  a  features  list  for  a  new  or  improved  system  or  product.  It 
identifies  the  scope  of  the  features  and  their  priority  to  provide  an  edge  in  the  market. 

b)  Maintenance  Concept.  This  focuses  on  life  cycle  logistics  goals,  objectives,  constraints,  and 
general  support  capabilities  related  to  a  desired  system  or  product. 

c)  Operations  Concept  (or  Concept  of  Operations).  This  focuses  on  the  goals,  objectives,  and 
general  desired  capabilities  of  a  potential  system  or  product  (new  or  improved),  without  indicating 
how  the  system  or  product  can  be  implemented. 

d)  Disposal  Concept.  This  focuses  on  the  planned  disposition  of  the  end  products,  and  by-products 
produced  throughout  the  life  cycle  of  the  system. 

e)  Request  for  Proposal  (RFP).  This  can  include  one  or  more  of  the  above  initiating  documents.  Its 
purpose  is  to  solicit  bids  for  consideration  from  several  sources  to  develop  a  system  or  product. 

2.  For  creation  activities  (system  definition,  subsystem  design,  detailed  design  and  integration,  and  test 
and  evaluation)  of  the  engineering  life  cycle,  source  documents  are  much  more  definitive  and  include 
one  or  more  of  the  following: 

a)  Contract.  This  type  of  negotiated  document  is  the  basis  for  most  project  efforts  involving  two 
enterprises.  It  often  includes  the  Operations  Concepts,  Maintenance  Concept,  Statement  of  Work, 
performance  specifications,  drawings,  and  interface  control  documents. 

b)  Statement  of  Work  (SOW).  This  provides  requirements  for  the  technical  work  to  be  accomplished 
by  an  assigned  team  or  project.  It  is  provided  as  part  of  a  contract  or  internal  tasking  document. 

c)  Tasking  Document.  This  is  a  type  of  an  agreement  between  two  parties,  typically  inside  an 
enterprise. 

D.2  Technical  Documents 

Technical  documents  are  dependent  on  the  applicable  enterprise-based  life-cycle  phase  and  describe  the 
technical  efforts  in  a  particular  area  to  be  accomplished  during  engineering  life  cycle  activities.  Technical 
documents  are  usually  prepared  by  the  project  during  an  earlier  enterprise-based  life  cycle  activity.  They  also 
can  be  included  in  source  documents  when  prepared  by  the  acquirer,  either  internal  or  external. 

Technical  documents  include  (alphabetically): 

1 .  Configuration  Management  (CM)  Plan.  This  document  defines  the  process  used  to  identify  and 
document  the  functional  and  physical  characteristics  of  the  system  during  its  life  cycle.  The  CM 
process  provides  a  means  of  controlling  changes  to  those  characteristics  and  provides  information  on 
the  status  of  changes.  (See  ANSI/EIA-649,  National  Consensus  Standard  for  Configuration 
Management.) 
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2.  Contractor  Integrated  Technical  Information  Services  (CITIS)  Plan.  This  document  describes  the 
methods  that  allow  access  and  delivery  of  required  digital  information  to  an  external  acquirer. 

3.  Data  Management  Plan.  This  document  reflects  the  data  requirements  of  an  agreement;  establishes 
data  management  criteria  and  responsibilities;  and  describes  the  enterprise  structure,  administration, 
and  control  procedures  used  to  ensure  effective  data  management  (internally  as  well  as  with  external 
acquirers  or  suppliers). 

4.  Electromagnetic  Compatibility/Interference  (EMC/EMI)  Control  Plan.  This  document  presents  the 
methods  that  allow  the  project  to  meet  the  EMC/EMI  requirements  related  to  the  system,  including,  as 
appropriate,  susceptibility  to  electromagnetic  pulse  from  nuclear  weapons.  It  communicates  the  work 
effort,  the  emphasis,  and  the  design  guides  to  be  used  in  avoiding  serious  electromagnetic 
compatibility  problems.  It  provides  guidance  to  assigned  teams  on  design,  specifications,  and 
installation  parameters  so  as  to  ensure  a  system  that  is  compatible  with  upper-layer  and  lateral  end 
products  and  enabling  products,  and  with  external  systems. 

5.  Engineering  Plan.  The  engineering  plan  provides,  to  project  personnel  and  the  acquirer,  the  planned 
technical  efforts  to  accomplish  the  processes  for  engineering  a  system  for  the  applicable  enterprise- 
based  life-cycle  phases  of  the  project.  The  engineering  plan  provides  (1)  an  understanding  of  the 
problem  to  be  solved,  (2)  what  is  planned  to  be  accopmplished,  (3)  how  it  will  be  done,  (4)  who  will 
do  it,  (5)  where  and  when  things  will  be  done,  and  (6)  resources  required,  including  when,  how  much, 
and  charcteristics.  The  focus  of  this  plan  is  on  risk  reduction.  This  plan  need  not  be  a  stand-alone 
document  but  can  be  part  of  the  project  plan.  In  military  projects,  this  plan  is  often  called  the  Systems 
Engineering  Plan  (SEP). 

6.  Human  Factors /Engineering  Plan.  This  document  focuses  on  human  factors  engineering  so  that  the 
best  human  performance  is  obtained  in  the  operation  of  the  highly  complex  equipment  developed  by  a 
project.  This  plan  is  built  on  the  assumption  that  the  capacities  of  humans  lie  within  certain  limits  and 
that  by  adapting  the  design  of  an  end  product  for  humans  requires  consideration  of  basic  human 
characteristics:  decision-making  capability;  muscular  strength  and  coordination;  body  dimensions; 
perception  and  judgment;  skills;  optimum  work  load;  and  requirements  for  safety,  comfort,  and 
freedom  from  environmental  stress. 

7.  Interface  Control  Plan.  This  document  identifies  and  defines  the  physical,  electronic,  and  content 
characteristics  of  all  system  internal  and  external  interfaces  and  communications  links.  It  ensures  that 
the  various  elements  of  the  system  are  functionally,  physically,  and  electronically  capable  of 
interacting  with  each  other,  and  with  all  external  links  with  which  they  must  connect  or  communicate, 
to  perform  required  functions.  This  includes  interfaces  with  people  as  well  as  hardware  and  software. 

8.  Supportability  Plan.  This  document  is  meant:  to  influence  the  end  product  design  solution  definition 
activities  to  consider  supportability  requirements;  to  identify  the  support  problems  and  items  that  drive 
the  cost  of  support  early  enough  to  change  the  design  to  fix  or  eliminate  the  support  problems;  to 
develop  a  complete  set  of  projections  of  all  resources  required  to  support  the  end  products  over  their 
life  time;  and  to  develop  and  use  a  single  database  for  all  analysis. 

9.  Maintenance  Plan.  This  document  emphasizes:  understanding  system  readiness  and  performance 
requirements,  physical  environments,  and  resource  availability  to  support  the  mission  and  purpose  of 
the  end  products;  managing  the  contributions  to  end  product  maintainability  that  are  made  by  enabling 
products;  developing  robust  end  products  that  are  insensitive  to  the  environment  experienced 
throughout  the  end  product’s  life  cycle  and  that  are  easily  repaired  under  adverse  conditions;  and 
determining  spares  requirements. 

10.  Producibility  Plan.  This  document  has  as  its  objective  the  achievement  of  a  producible  design  solution 
definition  at  the  lowest  possible  cost  while  maintaining  the  functional  integrity  and  quality  standards  of 
system  products.  It  includes  planning  for  the  analysis  and  coordination  of  internal- supplier  and 
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external- supplier  engineering,  manufacturing,  and  procurement,  and  provides  an  orderly  transition 
from  development  to  production.  Producibility  emphasizes  elimination  of  undesirable  production 
features  involving  number  of  parts,  materialism,  raw  material  forms,  fabrication  processes,  tooling,  and 
facilities. 

1 1 .  Reliability  Plan.  This  document  has  as  its  purpose  the  prevention,  detection,  and  correction  of  design 
anomalies,  weak  parts,  and  workmanship  defects. 

12.  Software  Development  Plan.  This  document  describes  a  developer’s  plan  for  conducting  a  software 
development  effort,  whether  for  a  new  development,  modification,  reuse,  reengineering,  maintenance, 
or  for  all  other  activities  resulting  in  software  products.  It  includes  the  software  development  process 
to  be  used,  the  activities  to  be  performed  in  each  software  build,  and  methods  to  be  used. 

13.  Specifications.  As  a  function  of  engineering  life  cycle  activities,  two  kinds  of  specifications  can  be 
available — performance  specifications  and  detail  specifications.  Performance  specifications  are 
outputs  of  the  Solution  Definition  Process  during  the  Pre-System  Definition  phase  of  the  engineering 
life  cycle  and  at  least  through  the  Subsystem  Design  phase.  Performance  specifications  generally  are 
stated  in  form,  fit,  and  function  terms.  They  can  designate  the  means  for  verifying  compliance.  Detail 
specifications  are  typically  an  output  of  the  detailed  design  activity,  especially  during  development  of 
lower-layer  building  block  product  designs.  Detail  specifications  generally  state  requirements, 
characteristics,  and  materials  related  to  a  specific  solution  or  approach,  thus  reducing  developer 
flexibility.  Both  kinds  of  specifications  can  be  included  in  a  government  contract  or  can  be  provided 
by  the  user,  prime  contractor,  or  another  project. 

14.  System  Safety  Plan.  This  document  has  the  objective  of  identifying,  evaluating,  eliminating,  or 
controlling  hazards  throughout  a  product’s  life  cycle.  This  plan  is  used  to  increase  safety  awareness 
within  assigned  teams  and  to  design  safety  into  end  products. 

15.  System  Security  Plan.  This  document  has  the  objective  of  identifying,  evaluating,  eliminating,  or 
controlling  security  concerns.  This  plan  is  used  to  increase  security  awareness  and  bring  about  the 
design  of  security  features  that  will  a)  reduce  an  organization’s  liability,  b)  address  privacy  issues,  and 
c)  correctly  assist  in  preserving  system  operations  and  maintain  system  integrity  when  accidental  or 
malicious  fault  events  occur. 

16.  Testability  Plan.  This  document  is  the  basic  tool  for  establishing  and  executing  an  effective  testability 
program.  This  plan  emphasizes:  integration  of  testability  requirements  with  other  design  requirements 
and  dissemination  to  assigned  teams  and  external  suppliers;  establishing  control  for  ensuring  that  each 
supplier’s  testability  practices  are  consistent  with  end  product  requirements;  identifying  testability 
design  guides  and  testability  analysis  models  and  procedures  to  be  used  by  teams;  planning  for  review, 
verification,  and  use  of  testability  data  submissions;  and  establishing  the  testability  tasks  that  are  to  be 
done,  how  each  task  is  to  be  done,  when  they  are  to  be  done,  and  how  the  results  of  the  tasks  are  to  be 
used. 

17.  Training  Plan.  This  document  establishes  the  personnel  and  training  requirements;  describes  the 
supplier-provided  training  courses  by  type  to  establish  skill  levels  to  effectively  perform  operations 
and  support  activities;  and  identifies  resources  and  supporting  actions  required  for  establishment  and 
support  of  the  training  courses. 

18.  Other  technical  plans.  The  above  list  of  technical  plans  is  not  meant  to  be  exhaustive.  This  Guide 
calls  for  other  plans  such  as  Verification  Plans,  Validation  Plans,  and  Test  Plans  for  which  much  of 
the  information  in  the  Testability  Plan  would  be  included  for  any  one  or  a  group  of  specific  tests;  and  a 
Technical  Performance  Measurement  Plan  (TPM).  Other  technical  plans  that  can  be  applicable  to  a 
project  include:  Computer  Resource  Development  Plan,  Manufacturing  Plan,  Mass  Properties 
Management  Plan,  and  Test  and  Evaluation  Management  Plan  (TEMP). 
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D.3  Enterprise  or  project  Documents 


Enterprise  or  project  documents  provide  directive  and  constraining  inputs  to  the  Planning  Process.  These 
documents  include: 

1 .  Enterprise  Policies.  Policy  documents  provide  a  framework  for  decision  making  in  the  conduct  of  a 
project  and  the  engineering  of  systems.  Policies  establish  the  criteria  by  which  decisions  are  made  in 
planning  particular  areas  of  an  engineering  effort  as  well  in  implementing  an  engineering  effort.  For 
instance,  a  policy  could  state  that  this  Guide  must  be  used  for  planning  all  enterprise  project  activities; 
or,  that  engineering  efforts  are  to  be  accomplished  using  teams  within  the  project-organizing  structure; 
or,  that  projects  are  to  use  a  particular  automated  tool  to  accomplish  a  certain  task  or  set  of  tasks  within 
the  processes  for  engineering  a  system;  or,  the  frequency  of  reporting  progress  or  making  progress 
checks. 

2.  Enterprise  or  project  Procedures.  Procedure  documents  contain  the  recommended  processes, 
approach,  or  steps  to  be  taken  in  completing  an  agreement  for  engineering  a  system.  Examples  of 
procedures  are:  how  reports  are  approved;  or,  how  technical  reviews  are  planned,  conducted,  and 
closed;  or,  the  activities  involved  with  planning,  conducting,  and  reporting  qualifying  tests  or 
validations. 

3.  Project  Plan.  This  document  provides  the  considered  management  approach  to  meet  the  requirements 
of  an  agreement.  It  lays  out  resource  availability  as  a  function  of  time  and  other  key  development 
schedule  requirements.  It  also  provides  the  budget  over  the  projected  suppliers.  This  plan  establishes 
the  necessary  boundaries  for  the  engineering  plan  and  other  technical  plans.  In  military  projects,  this 
plan  often  takes  the  form  of  an  Integrated  Master  Plan(IMP)  and  Integrated  Master  Schedule  (IMS). 

4.  Resource  Management  Plan.  This  document  can  contain:  staffing  availability,  manpower  loading 
limitations,  delivery  schedule  dates,  facility  availability  dates,  capacity  restrictions,  and  use  of 
particular  materials  or  reusable  hardware  or  software  units.  These  constraints  provide  process  and 
design  limits,  based  on  enterprise  and  project  resource  availability  or  policies. 

5.  Risk  Management  Plan.  This  document  describes  the  project  aspects  of  risk  identification  (sources  and 
causes),  risk  characterization  (effects,  probabilities,  choices,  time  frame,  and  coupling),  risk 
prioritization  (greatest  harm,  greatest  effect,  and  time  urgency),  and  risk  aversion  (mitigation, 
avoidance,  transfer,  and  acceptance).  It  identifies  the  risk  management  functions  to  be  performed  by 
assigned  teams  and  by  supporting  analysts  and  specialists.  The  acceptable  levels  of  risk  for  a 
particular  enterprise-based  life-cycle  phase,  or  group  of  phases,  are  included. 

6.  Strategic  Plan.  This  document  provides  insight  into  the  projects  and  the  markets  the  enterprise  plans 
to  pursue  over  a  given  time  frame.  The  Strategic  Plan  establishes  the  desired  enterprise  direction,  key 
objectives,  strategies  for  attaining  the  objectives,  and  metrics  by  which  progress  toward  meeting 
objectives  is  measured.  It  presents  how  the  enterprise  plans  to  compete  to  obtain  a  competitive 
advantage  to  outperform  competitors.  Plans  for  an  engineering  effort  are  to  be  consistent  with  and 
support  the  strategic  plan. 

7.  Total  Cost  of  Ownership  Plan.  This  document  describes  the  time-phased  technical  efforts  required  to 
control  the  total  ownership  cost,  and  hence,  the  affordability,  of  a  system  under  development.  The 
ultimate  cost  of  a  system  and  its  products  is  locked-in  very  early  in  the  enterprise-based  life  cycle  and 
with  each  application  of  development  life-cycle  processes.  This  document,  therefore,  discusses  the 
enterprise’s  or  project’s  plan  for  equating  cost  with  performance  and  schedule  requirements  in 
evolving  the  system  design;  for  balancing  the  future  costs  of  production,  operation,  support,  training, 
and  disposal;  and  for  taking  active  measures  for  meeting  affordability  objectives.  Specifically,  the  cost 
of  personnel  and  consideration  of  system  complexity,  open  system  architectures,  reuse,  and  other  such 
cost-saving  approaches  are  included  in  the  plan. 
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Appendix  E  -  System  Technical  Reviews  (informative) 


The  system  technical  reviews  of  Table  E.l  are  related  to  engineering  life-cycle  phases  and  are  relevant  to  the 
system  element  of  applicable  building  block  developments.  They  are  not  directly  related  to  enterprise-based 
life-cycle  phases  (see  Appendix  B);  however,  technical  review  exit  criteria  include  satisfying  the  exit  criteria  of 
the  applicable  enterprise-based  life-cycle  phase. 

System  technical  reviews  for  a  building  block  development  can  be  formal  (i.e.,  required  by  the  external 
customer  agreement).  Incremental  technical  reviews  for  the  subsystems,  associated  processes,  and  end  products 
are  generally  informal,  not  requiring  external  customer  participation  on  the  reviewing  body,  and  are  normally 
conducted  prior  to  the  system  technical  review.  System  technical  reviews  for  lower-layer  building  block 
developments  are  generally  informal  unless  required  to  be  formal  in  an  agreement. 

Table  E.l  -  Sytem  technical  reviews 


PHASE 

ENGINEERING  LIFE-CYCLE-PHASE  REVIEWS 

Pre- System 
Definition 

An  alternative  system  review ,  if  applicable,  considers  all  concepts  looked  at  and  selects  a 
preferred  concept  for  further  development  that  has  the  potential  for  satisfying  identified 
stakeholder  requirements.  Assesses  progress  toward  converging  on  a  viable,  traceable  set 
of  system  technical  requirements  that  are  balanced  with  cost,  schedule,  and  risk. 

System 

Definition 

A  system  requirements  review  validates  that  the  set  of  stakeholder  requirements  is 
complete,  consistent  with  acquirer’s  intent,  and  understood  by  the  developer. 

A  system  definition  review  demonstrates  convergence  on  and  achievability  of  technical 
requirements  and  readiness  to  initiate  the  Subsystem  Design  Phase. 

Subsystem 

Design 

A  Subsystem  requirements  review ,  held  for  each  sub  system- layer  building  block 
development,  validates  that  the  set  of  assigned  and  other  local  stakeholder  requirements 
is  complete,  consistent  with  stakeholder’s  intent,  and  understood  by  the  developer. 

A  system  preliminary  design  review  for  each  subsystem  building  block  development 
confirms  that: 

a)  subsystem  building  block  specifications  have  been  defined  appropriately; 

b)  subsystem  building  block  end  product  designs  satisfy  requirements  assigned  from 
the  parent  building  block; 

c)  enabling  products  for  the  associated  processes  have  been  defined  adequately  to 
initiate  enabling  product  developments; 

d) the  approaches  planned  for  next  lower-layer  building  blocks  are  appropriately 
planned; 

e)  lower-layer  building  block  project  risks  are  identified,  and  mitigation  plans  are 
feasible  and  judged  to  be  effective. 

Detailed 

Design 

A  system  detailed  design  review  for  each  lower-layer  building  block  development 
demonstrates  that 

a)  specifications  and/or  drawings  or  software  development  files  have  been 
appropriately  defined; 

b)  the  building  block  end  product  designs  satisfy  requirements  assigned  from  the 
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parent  building  block; 

c) 

enabling  products  for  the  associated  processes  have  been  defined  adequately  to 
initiate  enabling  product  developments; 

d) 

the  building  block  project  is  either:  (1)  ready  for  continued  development;  (2) 
appropriately  defined  for  purchase  of  products  form  an  external  supplier;  (3) 
ready  for  fabrication  of  building  block  elements;  or  (4)  adequately  defined  so 
that  off-the-shelf  products  or  reuse  products  can  be  used  to  fulfill  product 
requirements  and  are  available  within  the  enterprise. 

End  Product 

Physical 

Integration, 

Test  and 
Evaluation 

Readiness  reviews  for  each  building  block  from  the  bottom  up  demonstrate  that  delivered 
end  products  from  lower-layer  building  blocks  have  been  validated,  or  that  validation 
tests  are  adequately  planned,  and  that  each  set  of  integrated  products  forms  a  composite 
end  product  verification  and  end  product  validation,  if  required. 

Audits  are  intended  to: 

a) 

demonstrate  that  end  product  verification  is  compliant  with  their  specified 
requirements  and  confirms  that  product  verification  outcomes  compare 
favorably  against  configuration  documentation:  (1)  drawings;  (2)  test 
procedures;  (3)  authorized  changes;  (4)  software  development  files;  and  (5) 
“as-built”  or  “as-coded”  documentation; 

b) 

confirm  that  the  “as-built  or  “as-coded”  configuration  has  been  favorably 
examined  against  its  configuration  documentation:  (1)  drawings:  (2)  bill  of 
materials;  (3)  specifications;  (4)  code  lists;  (5)  manuals;  (6)compliance  test;  or 
(7)  compliance  data. 

Additionally,  audits  confirm  that: 

a) 

products  have  been  built  to  drawings  and  satisfy  specifications; 

b) 

the  information  database  represents  the  work  products  of  the  building  block 
development; 

c) 

required  changes  to  previously  completed  specifications  have  been 
implemented;  and 

d) 

enabling  products  for  down-stream  associated  processes  are  available,  can  be 
executed,  and  meet  stakeholder  requirements. 

Process  reviews  demonstrate  that  development  of  enabling  products  for  associated 
processes  is  on  schedule,  and  that  designs  satisfy  related  end  product  needs.  Production 
readiness  reviews  and  test  readiness  reviews  are  examples  of  process  reviews 
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Appendix  F  -  Process  Relationships  (informative) 


The  generation  and  use  of  various  requirements  and  representations  are  introduced  in  Subsection  4.3.  These  are 
further  described  below.  Figure  F.l  shows  the  relationship  of  these  requirements. 


OTHER 

STAKEHOLDER 

REQUIREMENTS 


ACQUIRER 


REQUIREMENTS 


Figure  F.  1  -  Requirement  relationships 


Acquirer  requirements  come  from  a  customer  or  user  (including  operators,  where  applicable)  for  a  major  system 
such  as  an  aircraft,  automobile,  check  processor,  mail  sorter,  or  telecommunication  switch.  Acquirer 
requirements  also  come  form  a  developer  needing  subsystems  to  make  up  an  end  product  of  a  system  (see 
Subsection  6.2).  The  latter  are  identified  as  assigned  requirements  and  would  have  been  defined  by  a  prior 
application  of  the  System  Design  processes  of  Subsection  4.3. 

Other  stakeholder  requirements ,  when  added  to  the  acquirer  requirements,  make  up  a  set  of  stakeholder 
requirements  that  are  transformed  into  system  technical  requirements.  Stakeholder  and  system  technical 
requirements  are  identified,  collected,  or  defined  by  completing  the  Requirements  Definition  Process 
(Subsection  4.3). 

The  logical  and  physical  solution  representations,  derived  technical  requirements,  design  solution  and  specified 
requirements  are  defined  by  completing  the  Solution  Definition  Process  (Subsection  4.3). 

Stakeholder  requirements  (acquirer  and  other  stakeholder  requirements),  as  well  as  system  technical 
requirements  and  the  derived  technical  requirements,  differ  from  specified  requirements. 

1)  In  effect,  stakeholder  requirements  constitute  the  input  that  establishes  the  problem  to  be  solved.  Such 
requirements  can  be  considered  as  the  initial  specification  for  a  development  effort  or  as  a  set  of 
specified  requirements  for  procuring  an  off-the-shelf  item.  End  products  developed  or  purchased,  and 
that  are  to  be  or  that  have  been  delivered  to  an  acquirer,  are  validated  against  these  specifications  (see 
Sub-process  33). 
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2)  The  derived  technical  requirements,  logical  solution  representations,  and  system  technical 
requirements  reflect  intermediate  evolution  states  that  are  technical  in  nature,  are  validated,  and  are 
measurable.  The  design  solution  is  verified  against  these  requirements  as  reflected  by  the  selected 
physical  solution  representation  (see  Sub-process  30). 

3)  Specified  requirements  constitute  the  controlled  definition  of  the  finished  solution.  These 
requirements  have  two  roles  (see  Figure  F.2).  The  first  role  is  to  represent  the  build- to,  buy-to,  or 
assemble  and  integrate-to  specifications,  drawings,  parts  lists,  etc.,  that  describe  the  design  solution  of 
the  product  will  be  verified  (see  Sub-process  31).  The  second  role  is  to  represent  the  assigned 
requirements  to  be  used  to  develop  the  subsystems  of  the  end  products  that  require  further 
development. 


OTHER 

STAKEHOLDER 

REQUIREMENTS 


ACQUIRER 

REQUIREMENTS 

TRACE  TO 


t 


BIJTLDTNG  BLOCK 


SYSTEM 

TECHNICAL 


REQUIREMENTS 

ASSIGNED  TO 


I 


ASSIGNED  TO 


LOGICAL 

SOLUTION 


ASSIGNED  TO 


\ 


DERIVED 

TECHNICAL 


REQUIREMENTS 


PHYSICAL 
SOLUTION 
REPRESENTATIONS 


REPRESENTATIONS 

DRIVE  I  ^  ! 

_  DRIVE  /  SOURCE 

T  r  J  * 

TO  T 


ASSIGNED  TO 


DESIGN  SOLUTION 

I 

SPEC  IE  TED  BY 


SPECIFIED 

REQUIREMENTS 


NOTE:  End  product  specified 
requirements  are  used  for  (1) 
building,  coding,  or  buying  of 
products,  (2)  assembly  & 
integration,  and  (3)  end  product 
verification 


Subsystem  specified  requirements  become  assigned  requirements 
at  next  lower  layer  of  building  block  development. . . 


BUILDING  BLOCK 


BUILDING  BLOCK 


BUILDING  BLOCK 


STAKEHOLDER 
REQUIREMENTS  “ 


ETO  SYSTEM 

TECHNICAL 

■  1  1  It- 

OTHER  TRACE  TO  SYSTEM 

STAKEHOLDER  TECHNICAL 

—j  1  1 

OTHER  TRACE  TO  SYSTEM 

STAKEHOLDER  TECHNICAL 

REQUIREMENTS  ASSIGNED  TO 

ASSIGNED  TO  | 

LOGICAL  ASSIGNED  TO  PHYSICAL 

SOLUTION  SOLUTION 

REQUIREMENTS 

REQUIREMENTS  ASSIGNED  TO 

ASSIGNED  TO  |  ^ 

LOGICAL  ASSIGNED  TO  PHYSICAL 

SOLUTION  SOLUTION 

REQUIREMENTS 

REQUIREMENTS  ASSIGNED  TO 

ASSIGNED  TO  | 

LOGICAL  ASSIGNED  TO  PHYSICAL 

SOLUTION  SOLUTION 

1  /  1 

DRIVE  \  DRIVE  /  SOURCE  OF 

ASSIGNED  TO  J  j 

DERIVED  DESISN  SOLUTION 

TECHNICAL 

REQUIREMENTS  j 

SPECIFIED  BY 

1  /  1 

DRIVE  \  DRIVE  /  SOURCE  OF 

ASSIGNED  TO  J  j 

DERIVED  DESISN  SOLUTION 

TECHNICAL 

REQUIREMENTS  J 

SPECIFIED  BY 

1  /  1 

DRIVE  \  DRIVE  /  SOURCE  OF 

ASSIGNED  TO  J  j 

DERIVED  DESISN  SOLUTION 

TECHNICAL 

REQUIREMENTS  J 

SPECIFIED  BY 

Figure  F.2  -  Role  of  specified  requirements 

Enabling  product  requirements  are  generated  during  the  application  of  the  Requirement  Definition  and  Solution 
Definition  processes  to  the  system,  end  product,  and  subsystem  elements  of  the  building  block  (see  Subsection 
6.1).  These  requirements  are  not  shown  in  Figure  F.l  since  they  become  the  basis  for  another  building  block 
development  that  uses  these  requirements  as  assigned  requirements.  An  example  of  the  development  of  two  of 
the  seven  associated  process  enabling  products  is  shown  in  Figure  F.3.  The  relationships  between  the  types  of 
requirements  and  the  processes  and  process  requirements  of  this  Guide  are  shown  in  Figure  4.3c  of  Section  4.3. 
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Layer  N  Building  Block 


Figure  F.3  -  Development  of  enabling  products 
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Appendix  G  -  Engineering  Specialty  References 


ENGINEERING  SPECIALTY  TABLE 

TECHNICAL  DISCIPLINE 

REFERENCE 

Abbreviations 

ASME  Y14.100 

ASME  Y14.38M 

Acquisition  Logistics  Handbook 

MIL-HDBK-502 

Asynchronous  Transfer  Mode  (ATM) 

MIL-STD- 188-176 

Climatic  information 

MIL-HDBK-3 1 0 

CM  Munitions  &  Computer  Programs  (CANCELLED) 

MIL-STD-483 

Computer  aided  acquisition  and  logistics  support 

MIL-HDBK-59 

Configuration  Management  (CANCELLED) 

MIL-STD-973 

Configuration  Management  (CANCELLED) 

MIL-STD-2549 

Configuration  Management  Guidance 

MIL-HDBK-61 

Corrosion  prevention  and  control 

MIL-  HDBK-1250 

MIL-  HDBK-1568 

Cost  Engineering 

MIL-HDBK- 1 0 1 0  A 

Defense  Specifications 

MIL-STD-961 

Defense  STDS  &  HDBKS 

MIL-STD-962 

Design  to  Cost  (CANCELLED) 

MIL-STD-337 

Digital  Data  Bus 

MIL-STD- 1553B 

Digital  Nautical  Chart  (DNC) 

MIL-PRF-89023 

Electromagnetic  compatibility 

MIL-STD- 1541 

MIL-STD-464 

MIL-HDBK-23  7 

Electromagnetic  Emissions 

MIL-STD-461 

Electronic  Reliability  Design 

MIL-HDBK-3  3  8 

Electrostatic  Discharge 

MIL-STD- 1686 

Engineering  Drawing  Practices 

MIL-STD- 100 

Engineering  Management  (CANCELLED) 

MIL-STD-499 

Environmental  analysis 

MIL-STD-810 

Fiber  Optic  Data  Bus 

MIL-STD- 1773 

Grounding  for  Communications  Systems 

MIL-STD- 188- 124 

Human  Factors  Engineering 

MIL-STD- 1472 
MIL-HDBK-46855 

ID  Markings 

MIL-STD- 130 

Interface  Shipboard  Systems 

MIL-STD-1399 

Logistic  Support  Analysis 

MIL-STD- 1388 

Logistics 

MIL-  HDBK-502 
MIL-PRF-49506 

Maintainability 

MIL-  HDBK-470 

MIL-HDBK-7  9 1 

Marking  for  Shipment  &  Storage 

MIL-HDBK- 129 

MIL-STD- 129 

Microelectronics  Test  Methods 

MIL-STD-883 

Military  Training  Programs 

MIL-HDBK- 1379.1,  .2,  .3,  A 
MIL-PRF-29612 

Nondestructive  inspection 

MIL-HDBK-728 
MIL-HDBK-73 1 

Parts  control 

MIL-HDBK-965 

Statement  Of  Work  (SOW) 

MIL-HDBK-245 

214 


ENGINEERING  SPECIALTY  TABLE 

TECHNICAL  DISCIPLINE 

REFERENCE 

Printed  Wiring 

IPC-D-275 

IPC-2221 

Producibility 

MIL-HDBK-727 

Quality  (CANCELLED) 

MIL-STD-9858 

Quality  Assurance  Terms  &  Definitions 

ISO  1804 

ANSI  8402 

Reliability  Testing 

MIL-HDBK-781 

Reliability/ durability 

MIL-  HDBK-1530 

MIL-  HDBK-87244 

MIL-  HDBK-1798 

MIL-  HDBK  -2164 

Review  &  Audits;  Software 

MIL-STD-1521B 

Sampling  Procedures 

MIL-STD-105 

Software 

IEEE/EIA  ISO  12207.0,  .1,  .2 

Software  (CANCELLED) 

DOD-STD-2167 

DOD-STD-2168 

Software  Support  Environment 

MIL-HDBK- 1 467 

Standardization  Program  Requirements  (CANCELLED) 

MIL-STD-680 

Supportability 

MIL-  HDBK- 5 02 

Survivability 

MIL-  HDBK -1799 

MIL-  HDBK  -2069 
MIL-HDBK-3  3  6 

System  Safety  Engineering 

MIL-STD-882 

System  Security 

MIL-  HDBK -1785 

Tech  Manuals;  Data  Base 

MIL-PRF-87269 

Technical  Data  Packages 

MIL-DTL-3 1 000A 

Technical  Manuals 

MIL-STD-40051 

T  elecommunications 

MIL-STD-188 

Test  Equipment  (CANCELLED) 

MIL-STD-1364 

Test  Reports 

MIL-STD-831 

Testability 

MIL-  HDBK  -2165 

Thermal  design/analysis 

MIL-HDBK-25 1 

Timing  &  Sync 

MIL-STD- 188-115 

Training  requirements 

MIL-  HDBK- 13  79 

Transportability 

MIL-STD-1366 

UHF  MILSATCOM  DAMA 

MIL-STD- 188- 185 

Vibrations 

MIL-STD- 167/2  A 

Weight  &  balance  control 

SAWE-RP7 

Work  Breakdown  Structure 

MIL-HDBK-881 
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Appendix  H  -  Naval  Process  Flow  Diagrams 


The  following  are  33  summary  charts  of  the  Naval  process.  Elaboration  and  details  (hyperlinks)  are  contained 
in  the  main  document. 
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Preceding  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 
Sub-process  8 


Sub-process  1:  Product  Supply 


Inputs 

•  Acquisition  Strategy  (SP  2) 

•  Solicitation  (RFP/SOO/SOW)  (SP  2) 

•  Acquirer  Offer  (SP  2) 

•  Requests  for  clarification  (SP  2) 

•  RFI  (SP  2) 

•  Acquirer  Signed  Agreement  (SP  2) 

•  NAVAIR  Specific: 

•  TAA  (SP  8) 


Purpose 

Establish  and  satisfy  an  agreement  with  the  acquirer. 


Tasks 


a)  Assess  the  acquisition  request,  offer,  or  directive  to 
determine  the  capability  to  meet  the  acquisition  document 
requirements. 

b)  Establish  a  satisfactory  agreement  within  legal,  regulatory, 
enterprise,  and  project  bounds. 

c)  Record  the  established  agreement  in  the  form  appropriate 
to  the  effort. 

d)  Implement  the  processes  of  this  Guide,  as  applicable,  to 
meet  the  requirements  of  the  agreement. 

e)  Deliver  the  products  and  other  deliverables  as  specified  in 
the  established  agreement. 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents. 


Agents 

Tools 

Contracts 

Requirements  Management  & 

Systems  Engineering 

System  Architecture  Data  Base 

-  Logistics/R&M 

Make  versus  Buy 

Business  Development 

PR  Web 

Acquirer 

Manufacturing 

Technical  Writer 

Legal 

Security 

Next  Process 

Acquisition  Process 
Sub-process  2 
Control  Process 
Sub-process  12 
Implementation  Process 
Sub-process  20 
System  Verification  Process 
Sub-processes  31,  32 
End  Product  Validation  Process 
Sub-process  33 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Supplier  Proposal  (SP  2) 

•  Supplier  Signed  Agreement  (SP  2) 

•  End  Products  (SP  2,  3,  20,  31 , 33) 

•  Enabling  Products  (SP  2,  3,  20,  31 ,  32, 
33) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 


References 

Standard  References 

OSD  Commercial  Item  Acquisition:  Considerations  and  Lessons 
Learned,  26  June  2000 
MIL-STD-961 ,  MIL-HDBK-245 

SD-2  Buying  Commercial  and  Non-Developmental  Items:  A  Handbook 
SD-5  Market  Research 

Managing  Quality  and  Productivity  in  Aerospace  and  Defense,  Nov 
1989 

NAVAIR  Specific:  NAVAIRINST  5400.154 


Metrics  and  Measures 

Timeliness 

Technical  performance  requirements  compliance 
Controlling  cost 
Management  Responsiveness 
Program  /  Subcontract  Management 
Discrepancies  (noted  and/or  unresolved) 
Acceptance  of  test  results 
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Preceding  Process 

Supply  Process 
Sub-process  1 
Acquisition  Process 
Sub-process  3 
Planning  Process 

Sub-processes  5,  6,  7,  8 
Requirements  Definition  Process 
Sub-process  14 
Solution  Definition  Process 
Sub-process  19 

End  Products  Validation  Process 
Sub-process  33 


Sub-process  2:  Product  Acquisition 


Inputs 

•  Supplier  Proposal  (SP  1) 

•  Supplier  Signed  Agreement  (contract  or 
program  directive)  (SP  1) 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Supplier  Performance  Management  Plan 
(SP  3) 

•  WBS  (SP  5) 

•  IMS  (SP  6) 

•  TEMP  (SP  7) 

•  SSP  (SP  7) 

•  TWP  (SP  8) 

•  SOO  (SP  8) 

•  SOW  (SP  8) 

•  ICD-  formerly  MNS  (SP  14) 

•  ODD  or  CPD  -  formerly  ORD  (SP  14) 

•  Specified  Requirements  (SP  19) 

•  OTRR  certification  message  (SP  33) 

•  Cost,  Schedule,  and  Performance 
constraints  (EXT) 

•  Acquisition  Strategy  (EXT) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


Purpose 

Establish  an  agreement  with  that  supplier. 


Tasks 


a) 


b) 

c) 

d) 

e) 

f) 


Prepare  the  applicable  acquisition  request,  offer,  or 
directive  to  obtain  supply  of  work  or  delivery  of  desired 
system  products. 

Evaluate  supplier  response  to  acquisition  request,  offer,  or 
directive. 

Make  offer  or  provide  directive  to  desired  supplier. 
Negotiate  agreement  to  establish  a  satisfactory  agreement 
within  legal,  regulatory,  enterprise,  and  project  bounds. 
Record  the  established  agreement  in  the  form  appropriate 
to  the  effort. 

Accept  delivered  products. 


Agents 

Tools 

Contracts 

Specifications 

Source  Selection 

PRWeb 

Legal 

Proposal  Evaluation  Report 

Program  Manager  (PM) 

Turbo  Streamliner 

System  Engineering 

Logistics 

T&E 

Turbo  Specright! 

•  Next  Process 

Supply  Process 
Sub-process  1 
Acquisition  Process 
Sub-process  3 
Planning  Process 

Sub-process  5,  6,  8 
Control  Process 
Sub-process  12 
Transition  to  Use  Process 
Sub-process  21 _ 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Cost,  schedule,  and  performance 
constraints  (SP  5,  8) 

•  Acquisition  Strategy  (SP  1 , 5,  6) 

•  Solicitation  (RFP,  SOW  or  SOO  with 
Cost/Schedule  Requirements)  (SP  1,  3,  5) 

•  Acquirer  Offer  (SP  1 ) 

•  Request  for  Clarification  (SP  1) 

•  Request  for  Information  (RFI)  (SP  1) 

•  Acquirer  Signed  Agreement  (contract  or 
program  directive)  (SP  1 ,  3) 

•  ILS  Certification  (SP  21 ) 

•  Signed  DD  Form  250(s)  (SP  21) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 

MIL-STD-499B,  MIL-HDBK-245,  MIL-STD-961D 
Managing  Quality  and  Productivity  in  Aerospace  and  Defense,  Nov  1989 
OSD  Commercial  Item  Acquisition:  Considerations  and 
Lessons  Learned,  26  June  2000 

SD-2  Buying  Commercial  and  Non-Developmental  Items:  A  Flandbook 
SD-5  Market  Research 
CMMI  2001 

DD  Forms  1423,  250,  254 


Metrics  and  Measures 

IPT  Participation,  Review  and  Concurrence 
Technical  Reviews 
Product  Metrics 
Process  Metrics 

Measures  of  Effectiveness  and  Suitability 

Measures  of  Performance 

Technical  Performance  Measurements 


Suitability  Metrics 
Product  Affordability 
Timing 

Earned  Value 
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Preceding  Process 

Supply  Process 


Sub-process  3:  Supplier  Performance 


Next  Process 


Sub-process  1 

Acquisition  Process 

Sub-process  2 

Assessment  Process 

Sub-process  9,  10 

Control  Process 

Sub-process  13 

Solution  Definition  Process 
Sub-process  19 

Purpose 

Manage  supplier  performance  (and  sub-suppliers)  to  ensure  that 
the  technical  effort  to  be  accomplished  by  the  supplier  provides 

Acquisition  Process 

Sub-process  2 

Control  Process 

Sub-process  12 

end  products  that  satisfy  the  assigned  requirements. 

Tasks 

a)  Define  the  required  developer-supplier  relationships 

b)  Participate  on  appropriate  supplier  product  teams. 

c)  Monitor  supplier  performance  against  key  product  metrics. 

d)  Flow-down  changes  in  requirements  or  operational  concept 
that  might  affect  the  supplier’s  project. 

e)  Control  changes  to  requirements  made  by  the  supplier  that 
would  affect  the  developer’s  project  or  other  related  projects 
or  products. 

f)  Assess  supplier  performance  against  assigned  requirements 
including  conduct  of,  or  participation  in,  appropriate  technical 
reviews. 

g)  Validate  products  delivered  from  the  supplier,  or  ensure  that 
products  have  been  validated  before  delivery  and  prior  to 
integration  with  other  products  that  form  a  composite  end 
Droduct  intended  to  meet  the  develoDer’s  SDecified 

Outputs 

All  outputs  should  be  archived  SP  12 
•  Supplier  Performance  Management 
Plan  (SP  2,  3) 

Inputs 

•  Solicitation  (RFP,  SOW  or  SOO  with 
Cost/Schedule  Requirements)  (SP  2) 

•  Specified  Requirements  (SP  19) 

•  Acquirer  Signed  Agreement  (contract 
or  program  directive)  (SP  2) 

•  Approved  changes  (SP  13) 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Plans  and  schedules  trend  analysis 
(SP  9) 

•  Requirements  trend  analysis  (SP  10) 

requirements. 

Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 

•  Key  metrics  have  been  met. 

•  Tech,  reviews  have  been 
completed. 

•  Delivered  Product  satisfies 
requirements  and  approved  changes. 

Entrv  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 

•  Sponsor/User  Agreement 

•  Negotiated  Agreement 

•  Validated  Requirements 

Agents 

Acq  u  i  re  r/Develo  per 

PEO/PM 

User/Fleet 

Logistics 

Procurement 

Systems  Engineering 

Tools 

Requirements  Management  & 
System  Architecture  Data  Base 
Project  Management  Tools 

Tools  Survey:  Requirements 
Management  Tools 

References 

Standard  References 

CMMI  2001 

Metrics  and  Measures 

Report  supplier  progress  agains 
Report  percentage  of  Flow  Dow 
Report  on  percentage  of  produc 
validated 

it  Key  Metrics 

n  requirements  changes  (CM) 

:ts  delivered  that  have  been 
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Next  Process 

Planning  Process 


Purpose 

Define  a  strategy  for  implementing  the  adopted  process  of  this 
Guide  as  a  basis  for  project  technical  planning  and  that  is  in 
accordance  with  the  agreement. 


Tasks 

a)  Identify  stakeholders  who  will  have  an  interest  or  stake  in  the 
outcome  of  the  project. 

b)  Identify  and  acquire  applicable  documents  and  the  requirements 
therein,  that  could  affect  the  project. 

c)  Identify  process  approaches  required  to  develop  enabling  products 
(e.g.,  test,  training,  etc.). 

d)  Identify  applicable  enterprise-based  life-cycle  phases,  expected  work 
product  outputs,  applicable  management  reviews,  and  life-cycle- 
phase  exit  criteria. 

e)  Identify  and  define  how  the  applicable  processes  of  this  Guide  will  be 
integrated,  how  internal  and  external  projects  will  be  involved,  and 
how  they  will  be  integrated. 

f)  Identify  and  define  progress  assessment  metrics  and  reporting 
requirements. 

g)  Prepare,  document,  and  make  available  the  process  implementation 
strategy. 


Sub-process  5,  6,  7,  8 
Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  15 


All  outputs  should  be  archived  SP  12 

•  List  of  stakeholders  and  roles  (SP  4,  15) 

•  Associated  process  approaches  (SP  4) 

•  Life-cycle  phase  chart  (Milestones)  (SP  4,  6, 

8) 

•  Work  products  and  outputs  (SP  4) 

•  Work  product  reviews  (SP  4) 

•  Life-cycle  phase  exit  criteria  (SP  4,  8) 

•  List  of  applicable  tasks  (SP  4) 

•  Program  metrics  and  reporting  requirements 
(SP  4) 

•  Project  Library  (SP  5) 

•  Process  Implementation  Strategy  (SP  5,  6,  7, 

_ 8) _ 

Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agent. 

Planning  team  agrees  to  estimates  and 
customers  acknowledge  receipt  of 
information. 


References 

Standard  References 
CMMI  2001 
NAVAIR  Specific: 

NAVAIR  Acquisition  Guide 
NAVAIRINST  4200.36C,  Acquisition  Plans 
Class  Desk  Orientation,  March  2000 


Metrics  and  Measures 

Estimated  cost  of  project 
Estimated  schedule  of  project 
Estimated  cost  and  time  spent  planning 
Actual  cost  and  time  spent  planning 
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Preceding  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 

Sub-process  4,  6,  7 
Requirements  Definition  Process 
Sub-process  14,  15,  16 


Sub-process  5:  Technical  Effort  Definition 


Inputs 

•  Process  Implementation  Strategy  (SP  4) 

•  Project  Library  (SP  4) 

•  Organizational  Structure  (SP  6) 

•  IMS  (SP  6) 

•  Acquirer  requirements  (SP  14) 

•  MOEs  (SP  14) 

•  Other  stakeholder  requirements  (SP  15) 

•  System  technical  requirements  (SP  16) 

•  Data  and  Document  Management  Plans 
(SP  7) 

•  Configuration  Management  Plans  (SP  7) 

•  Acquisition  Strategy  (SP  2) 

•  Cost,  schedule,  and  performance 
constraints  (SP  2) 

•  Solicitation  (RFP,  SOW  or  SOO  with 
Cost/Schedule  Requirements)  (SP  2) 

NAVAIR  Specific:  POG  (SP  6) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 


Purpose 

Define  a  technical  effort  that  is  in  accordance  with  the  process 
implementation  strategy. 


Tasks 


a) 

b) 

c) 

d) 

e) 

f) 

g) 

h) 


Identify  project  tasks  to  include  all  requirements;  and  enterprise, 
project,  and  associated  process  constraints. 

Establish  an  Enterprise  Data  Repository  (including  an 
information  database). 

Determine  the  risk  management  strategy. 

Define  product  metrics  and  process  metrics. 

Establish  cost  objectives  to  be  used  in  Trade-off  analyses. 
Identify  technical  performance  measures  that  will  receive 
management  focus  and  be  tracked  using  TPM  procedures. 
Identify  applicable  tasks. 

Identify  the  appropriate  methods  and  tools,  required  facilities 
and  equipment,  and  training  required. 

Identify  applicable  or  potential  technology  constraints  and 
develop  an  approach  for  overcoming  each  constraint. 


Agents 

Tools 

Acquirer 

WBS 

PEO/PM 

EVMS 

End  User 

Systems  Engineering 

Technical  Writer 

Next  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 
Sub-process  6,  7,  8 
Assessment  Process 
Sub-process  9,  10,  11 
Control  Process 

Sub-process  12 

Requirements  Definition  Process 
Sub-process  16 
Systems  Analysis  Process 
Sub-process  22,  24 
System  Verification  Process 
Sub-process  32 


Outputs 

All  outputs  should  be  archived  SP  12 

•  TDP  (SP  16) 

•  Enterprise  Data  Repository  (SP  12) 

•  Risk  Mngmnt  Strategy  (incl.  Risk  Advisory  Board 
reqt(s)  (SP  24) 

•  Program  metrics  (SP  9) 

•  Process  metrics  (SP  9) 

•  Product  metrics  (SP  10) 

•  Testing  metrics  (SP  11) 

•  CAIV  decision  criteria  (SP  22) 

•  Total  Life  Cycle  Cost  Objectives  (SP  6,  8) 

•  Tech  Perf  Measures  (TPM)  (SP  9,  1 0,  1 1 , 22) 

•  WBS  (with  WBS  Dictionary)  (SP  2,  6,  7,  9,  10) 

•  Inputs  to  EVMS  (SP  8,  9) 

•  Technology  Roadmap  (SP  16) 

•  List  of:  Methods  and  Tools,  Facilities,  Equipment, 
Training  (SP  32) 

•  Life  Cycle  Support  Plans  (SP  16) 

•  Pre-Plan  Prod.  Improvement  (P3I)  (SP  16) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 


References 

Standard  references 
Top  Eleven  Ways  to  Manage  Risk 
CMMI  General  Information 
Business  Case  Analysis  Risk  Assessment  Matrix 
Managing  Quality  and  Productivity  in  Aerospace  and 
Defense,  Nov  1989 

OSD  Commercial  Item  Acquisition:  Considerations  and 
Lessons  Learned,  26  Jun  2000 


AVDEP-HDBK-7,  Rev.1,  1  Feb  1996 
EIA-748  EVMS,  1998 
MIL-HDBK-881 , 2  Jan  1998 
ISO  9000  and  ISO  14000 
NAVAIR  Specific: 

Web  addresses  cited  for  DAU  EVM  Dept  /  EVMS 
Policy  /  EVM  Implementation  Guide 
Draft  NAVAIR  Risk  Management  Guide 


Metrics  and  Measures 

Risk  Cube 

EVMS 

WBS 

Capability  Maturity 
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Preceding  Process  ^ 

Acquisition  Process  ” 

Sub-process  2 
Planning  Process 
Sub-process  4,  5 
Requirements  Definition  Process 
Sub-process  16 


Inputs 

•  Acquisition  strategy  (SP  2) 

•  Total  Life  Cycle  Cost  Objective 
(SP  5) 

•  System  Technical  Requirements 
(SP  16) 

•  WBS  (SP  5) 

•  Life  Cycle  Phase  Chart 
(Milestones)  (SP  4) 

•  Process  Implementation  Strategy 
(SP  4) 


Sub-process  6:  Schedule  and  Organization 


Purpose 

To  schedule  and  organize  the  defined  technical  effort 

Tasks 


a) 


b) 


c) 

d) 


e) 


Develop  an  event-based  schedule  that  integrates  key  events 
(internal  and  external),  related  tasks,  specialty  engineering 
tasks,  and  relevant  completion  criteria  for  the  applicable 
enterprise-based  life-cycle  phase. 

Develop  the  calendar-based  schedule,  showing  the  dates  of 
expected  task  and  event  completions  and  the  dependency 
relationships  among  tasks,  with  the  goal  of  developing 
information  for  an  Integrated  Master  Schedule  (IMS). 

Identify  resources  required  to  complete  scheduled  tasks. 

Define  the  staffing  and  discipline  needs  to  complete  the 
scheduled  tasks,  training  needs,  and  risks  if  required  staff  is  not 
available. 

Define  the  team  and  organizational  structure  to  complete  the 
scheduled  tasks  within  resource  constraints. 


Next  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 

Sub-process  5,  7,  8 
Assessment  Process 
Sub-process  9,  10,  11 
Control  Process 
Sub-process  12 
Transition  to  Use  Process 
Sub-process  21 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Resource  requirements  (staffing,  cost) 
(SP  7) 

•  Organizational  structure  (SP  5,  7,  8) 

•  IMS  (SP  2,  5,  7,  8,  9,  10,  11,21) 
NAVAIR  Specific: 

•  Program  Operating  Guide  (POG)  (SP  5) 


Entry  Criteria 

Inputs  have  been  approved 
by  the  appropriate  agent. 

•  Milestone  approval 

•  Receipt  of  funding 

•  Request  from  Acquirer 


Agents 

Acquirer 

Systems  Engineering 
User 

Specialty  Engineering 
Logistics 


Tools 

Scheduling  tools 
Estimating  tools 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agent. 

•  All  task  and  work  allocated  plus 
resources  identified. 

•  Firm  organizational  structure 


References 

Standard  References 

DI-MISC-81 183A  Integrated  Master  Schedule  Data  Item  Description 
(DID) 

CMMI  2001 


Metrics  and  Measures 

Schedule  variance  (SV) 
Cost  variance  (CV) 
Staffing 

Percent  complete 
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Preceding  Process 

Planning  Process 

Sub-process  4,  5,  6 
Requirements  Definition  Process 
Sub-process  14 


Inputs 

•  ICD  -  formerly  MNS  (SP  14) 

•  CDD  or  DPD  -  formerly  ORD  (SP 
14) 

•  MOEs  (SP  14) 

•  Process  Implementation  Strategy 
(SP  4) 

•  WBS  (SP  5) 

•  IMS  (SP  6) 

•  Organizational  Structure  (SP  6) 

•  Resource  Requirements  (SP  6) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 

•  Key  milestones  established 

•  Technical  effort  and  organization 
defined 


References 
Standard  References 
DI-MGMT-81024,  Systems  Engineering 
Management  Plan  (SEMP) 
DI-NDTI-80566 
DI-MISC-81 180 
CMMI  2001 
EVMS 

MIL-  HDBK-512A,  MIL-HDBK-61A 
IEEE  1220 


Sub-process  7:  Technical  Plans 


Next  Process 


Purpose 

Create  technical  plans  to  ensure  an  integrated  and  cost  effective 
technical  effort  in  accordance  with  the  defined  schedule  and 
organization. 

Tasks 

Prepare  appropriate  plans.  Plans  to  consider  include  the  following: 

a)  Engineering  Plan 

b)  Risk  Management  Plan 

c)  Technical  Review  Plan 

d)  Verification  Plans 

e)  Validation  Plans 

f)  Other  applicable  plans  as  called  for  in  the  agreement  or  by 
enterprise  policies  and  procedures 


Agents 

Acquirer 

Systems  Engineering 

Program  Manager 

Test  Engineers 

COMOPTEVFOR 

Contractors 

Tools 

Planning  and  scheduling  tools 
Automated  Systems 
Engineering  tools 

Metrics  and  Measures 

DI-CMAN-80858B 

ISO/IEC  12207 

ISO  9001 

DAU:  Risk  Management  Guide  for  DoD 
Acquisition 

NAVAIR  Specific: 

NAVAIR  INST  4130. 1C 

NAVAIR  INST  5000.21 

NAVAIR  INST  4355. 19B 

Plans  completed  and 
released  on  time 

APMSE  Quick  Reference  Guide 


Acquisition  Process 
Sub-process  2 
Planning  Process 
Sub-process  5 
Assessment  Process 

Sub-process  9,  10,  11 
Control  Process 

Sub-process  12 
Implementation  Process 
Sub-process  20 
Systems  Analysis  Process 
Sub-process  22,  24 
Requirements  Validation  Process 
Sub-process  25,  26,  27,  28,  29 
System  Verification  Process 
Sub-process  30,  31 
End  Product  Validation  Process 
Sub-process  33 


Outputs 

All  outputs  should  be  archived  (SP  12) 

•  SEP  and/or  SDP  (SP  9,  1 0,  1 1 , 22,  24,  30) 

•  TEMP  (SP  2,  11,  30,  31, 33) 

•  Risk  Management  Plan  (SP  24) 

•  CRLCMP  (SP  9) 

•  Configuration  Mngmnt  Plan  (SP  5,  9) 

•  QA  Program  Plan  (SP  20) 

•  Manufacturing  Plan  (SP  20) 

•  Data  and  Document  Mngmnt  Plan  (SP  5,  13) 

•  Security  Mngmnt  Plan  (SP  ALL) 

•  Verification  Plan  (incl.  the  Verification  Compliance 
Req.  Matrix  (VCRM))  (SP  25,  30,  31) 

•  Validation  Plan  (to  incl.  what  NAVAIR  calls  Op. 

Test  Plan  and  Dvlpmnt  Test  Plan),  known  as  OTTR 
Plan  in  SP  33  (SP  1 1 , 25,  26,  27,  28,  29,  33) 

•  IV  and  V  Plan  (SP  30) 

•  SSP  (SP  2) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 
All  technical  plans  identified,  written, 
and  approved. 
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Preceding  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 

Sub-process  4,  5,  6 
Requirements  Definition 
Sub-process  16 


Sub-process  8:  Work  Directives 


Inputs 

•  Process  Implementation  Strategy 
(SP4) 

•  Life  Cycle  Phase  Chart  (SP  4) 

•  Total  Life  Cycle  Cost  Objectives 
(SP  5) 

•  Life-cycle  phase  exit  criteria  (SP  4) 

•  Organizational  Structure  (SP  6) 

•  IMS  (SP  6) 

•  Inputs  to  EVMS  (SP  5) 

•  Cost,  schedule,  and  performance 
constraints  (SP  2) 

•  System  Technical  Requirements 
(SP  16) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents. 


Purpose 

Create  work  directives  that  implement  the  planned  technical  effort. 


Tasks 

a) 


b) 


Develop  individual  project  team  or  organization  work 
packages  that  describe  the  work  to  be  done,  resource 
sources,  schedules,  budget,  and  reporting  requirements. 
Generate  work  authorizations  for  the  team  or  organization 
that  provide  approval  for  applicable  teams  or  organizations  to 
complete  their  work  package  requirements  and  to  release 
applicable  resources. 


Agents 

Tools 

Acquirer,  PEO/PM,  IPT,  Systems 

WBS 

Engineering,  Logistics 

TAA  Form 

Next  Process 

Supply  Process 
Sub-process  1 
Acquisition  Process 
Sub-process  2 
Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  15 
System  Verification  Process 
Sub-process  30 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Team  Work  Plan  (TWP)  (SP  2,  15, 

30) 

•  SOO  (SP  2,  15,  30) 

.  SOW  (SP  2,  15,  30) 

•  NAVAIR  Specific: 

•  TAA  (SP  8) 


Exit  Criteria 

Outputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents. 

(TAA  Signed,  WBS  defined) 


References 

Standard  References 
EVMS  Industry  Standards  (EIA-748) 
MIL-HDBK-881 ,  MIL-STD-245D 
NAVAIR  Specific: 

NAVAIRINST  5400.154 


Metrics  and  Measures 

Risk  Cube 

EVMS 

WBS 

Capability  Maturity 
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Preceding  Process 

Planning  Process 

Sub-process  5,  6,  7 
Systems  Analysis  Process 
Sub-process  23 


Sub-process  9:  Progress  Against  Plans  and  Schedules 


Purpose 

Assess  the  progress  of  the  program  effort  against  applicable  plans, 
schedules,  and  budgets. 


Inputs 

•  TPM  (SP  5) 

•  WBS  (SP  5) 

•  Inputs  to  EVMS  (SP  5) 

•  Program  metrics  (SP  5) 

•  Process  metrics  (SP  5) 

•  IMS  (SP  6) 

•  SEP  or  SPD  (SP  7) 

•  CRLCMP  (SP  7) 

•  Configuration  Management  Plan 
(SP  7) 

•  Trade-off  Analysis  Technical 
Report  (SP  23) 


Entry  Criteria 

Inputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents 


Tasks 

a)  List  the  appropriate  events  such  as  system  specification,  design 
reviews,  tasks,  and  process  metrics,  including  capability 
maturity,  for  monitoring  progress  against  plans  and  schedules. 

b)  Collect  and  analyze  identified  process  metrics  data  and  results 
from  completion  of  planned  and  scheduled  tasks  and  events. 

c)  Compare  process  metrics  data  against  plans  and  schedule 
using  trend  analysis  to  determine  technical  areas  requiring 
management  or  team  attention. 

d)  Determine  risk  and  identify  need  to  correct  variances,  make 
changes  to  plan  and  schedule,  and  redirect  work  because  of 
risk. 


Aqents 

Tools 

Acquirer 

Rqmts  Mgmt  &  Sys  Arch  Db 

Stakeholder 

Schedule  software  w/lnsight 

Program  Management 

Completion  Date  Histogram 

Systems  Engineering 

Logic  Diagrams 

Logistics 

Gantt  Bar  Charts 

Cost 

Milestone  Charts 

Resource/Hour  Usage  Charts 

Earliest,  Expected  and  Latest 

Completion  Dates  and 

Durations 

Next  Process 

Acquisition  Process 
Sub-process  3 
Assessment  Process 
Sub-process  1 1 
Control  Process 
Sub-process  12 
Systems  Analysis  Process 
Sub-process  23,  24 


Outputs 

All  outputs  should  be  archived  SP  12 

•  List  of  appropriate  events,  tasks,  and 
process  metrics  (SP  9) 

•  Process  metrics  data  (SP  9) 

•  Program  metrics  data  (SP  9) 

•  Plans  and  Schedule  Trend  Analysis 
(SP  3,  9,  11,  12,23,  24) 

•  Cost  Performance  Report  (CPR  or 
C/SSR)  (SP  12) 


Exit  Criteria 

Outputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents 


References 

Standard  References 

DAU  Program  Manager’s  Tool  Kit 

MIL-STD-499B  Systems  Engineering 

EVMS 

CMMI,  2001 

NAVAIR  Specific: 

NAVAIR  Acquisition  Guide 
NAVAIRINST  4355.1 9B 


Metrics  and  Measures 

Percent  EVMS  that  is  not  level  of  effort 
Accuracy  of  trend  analysis 

Amount  of  time  between  the  closing  of  a  reporting  period  and  the 
reporting  of  a  metric 

Number  of  team  members  that  have  access  to  their  appropriate 
metrics 

IPT  member  satisfaction  with  the  metrics 
Provided  EVMS  metrics  used 
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Preceding  Process 

Planning  Process 

Sub-process  5,  6,  7  ““H 

Assessment  Process 
Sub-process  1 1 

Requirements  Definition  Process 
Sub-process  14,  16 
System  Verification  Process 
Requirements  30,  31 


Inputs 

•  CDD  -  formerly  ORD  (SP  14) 

•  ICD  -  formerly  MNS  (SP  14) 

•  SEP  or  SDP  (SP  7) 

•  TPM  (SP  5) 

•  WBS  (SP  5) 

•  Key  Performance  Parameters 
(SP  16) 

•  Product  metrics  (SP  5) 

•  IMS  (SP  6) 

•  Technical  Review  Report 
(SP  11) 

•  Design  Solution  Deficiency  & 
Discrepancy  Reports  (SP  30) 

•  End  Product  Deficiency  & 
Discrepancy  Reports  (SP  31) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Sub-process  10:  Progress  Against  Requirements 


Purpose 

Assess  the  progress  of  system  development  by  comparing 
currently  defined  system  characteristics  against  requirements. 


Tasks 


a) 


b) 

c) 

d) 


e) 


Identify  product  metrics,  and  their  expected  values,  that  will  affect 
the  quality  of  the  product  and  provide  information  of  the  progress 
toward  satisfying  acquirer  and  other  stakeholder  requirements,  as 
well  as  derived  requirements. 

Collect  and  analyze  product  metrics  data. 

Record  rationale  for  decisions  and  assumptions  made  with 
respect  to  collected  data. 

Compare  results  against  requirements  to  determine  degree  of 
technical  requirement  satisfaction,  progress  toward  maturity  of 
the  system  (or  portion  thereof)  being  engineered,  and  variations 
and  variances  from  requirements. 

Identify  deficiencies  and  discrepanciesto  specifications  and 
configuration  baselines. 


Agents 

Tools 

Program  Management 

Rqmts  Mgmt  &  Sys  Arch  Db 

Systems  Engineering 

Schedule  software  w/lnsight 

Logistics 

Cost 

Next  Process 

Acquisition  Process 
Sub-process  3 
Assessment  Process 
Sub-process  1 1 
Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  19 
Systems  Analysis  Process 
Sub-process  23,  24 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Requirement  trend  analysis 
(SP  3,  11,23,  24) 

•  Deficiencies  and  discrepancies  (SP 
11,  19) 


Exit  Criteria 

Outputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents 


References 

Standard  References 

DAU  Program  Manger’s  Tool  Kit 

DAU  Program  Manager’s  Tool  Kit 

MIL-STD-499B  Systems  Engineering 

EVMS 

CMMI  2001 

NAVAIR  Specific: 

NAVAIR  Acquisition  Guide 
NAVAIRINST  4355.1 9B 


Metrics  and  Measures 

Percent  requirements  (appropriate  to  the  level  of 
development)  that  have  been  analyzed,  and  percent 
deficiencies  and  discrepancies  identified  and  reported  to  the 
appropriate  agents. 
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Preceding  Process 

Planning  Process 

Sub-process  5,  6,  7 
Assessment  Process 
Sub-process  9,  10,  11 
Requirements  Definition  Process 
Sub-process  14,  16 
Solution  Definition  Process 
Sub-process  19 
System  Verification  Process 
Sub-process  30,  31 


Inputs 

•  CDD- formerly  ORD  (SP  14) 

•  ICD- formerly  MNS  (SP  14) 

•  Testing  metrics  (SP  5) 

•  TPM  (SP  5) 

•  IMS  (SP  6) 

•  Validation  Plan  (SP  7) 

•  SEP  or  SDP  (SP  7) 

•  TEMP  (SP  7) 

•  Plans  and  schedules  trend  analysis 
(SP  9) 

•  Requirement  trend  analysis  (SP  10) 

•  Deficiencies  and  discrepancies  (SP  10) 

•  System  Requirements  Document 
(SP  16) 

•  System  technical  requirements  (SP  16) 

•  Specified  requirements  (SP  19) 

•  Design  solution  deficiency  and 
discrepancy  reports  (SP  30) 

•  End  product  deficiency  and  discrepancy 
reports  (SP  31) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 


Sub-process  11:  Technical  Reviews 


Purpose 

Conduct  technical  reviews  of  progress  and  accomplishments  in 
accordance  with  appropriate  technical  plans. 

Tasks 

a)  Identify  the  review  objectives  and  requirements  cited  in  the  Systems 
Engineering  Plan  (SEP);  enterprise  policies  and  procedures;  and 
agreement,  as  applicable. 

Verify  completion  of  the  technical  review  entry  requirements. 
Establish  the  technical  review  board,  agenda,  and  speakers. 
Prepare  the  appropriate  materials  to  include  in  the  read-ahead 
technical  review  package  and  presentation  package. 

Facilitate  and  support  identification  and  resolution  of  emerging 
issues  prior  to  the  review. 

Conduct  the  technical  review  using  the  guidance  of  the  Design 
Review  Handbook  according  to  the  SEP,  identifying  and 
documenting  action  items  required  to  meet  the  review  objectives. 
Close  out  the  review  after  (1)  minutes  have  been  prepared, 
approved,  and  distributed;  (2)  action  items  have  been  resolved;  and 
(3)  the  review  has  been  signed-off  by  the  director. 


b) 

c) 

d) 


f) 


g) 


Agents 

Tools 

Acquirer 

Rqmts  Mgmt  &  Sys  Arch  Db 

Stakeholders 

Program  Management 

Systems  Engineering 

Logistics 

Next  Process 

Assessment  Process 
Sub-process  10 
Control  Process 
Sub-process  12 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Technical  Review  Report  (SP  10) 


Exit  Criteria 

Outputs  have  been 
reviewed  and  approved  by 
the  appropriate  agents 


References 

Standard  References 
MIL-STD-499B  Systems  Engineering 
NAVSO  P-6071  Best  Practices  Section  4.0 

DoD  4245. 7-M  Transition  from  Development  to  Production  Chapter  3 

MIL-STD-1521 

NAVAIR  Specific: 

NAVAIRINST  4355.19B 

NAVAIR  Design  Review  Handbook  (AIR  4.1) 


Metrics  and  Measures 

Minutes  and  action  items  completed  and  accepted  by  the 
appropriate  agent 
Functional  Allocation 
Performance 
Cost,  Schedule,  Weight 
Risk 
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Preceding  Process 

All  System  Engineering  Processes 
Sub-process  1-11,  13-33 


Inputs 

The  following  is  a  generalized  list,  not  all- 
inclusive,  of  information  that  should  be  included  in 
the  Enterprise  Data  Repository. 

•  Mission  Areas  (NMETL,  MCPs,  JTLs,  etc.) 

•  Solicitations 

•  Proposals 

•  Signed  agreements 

•  Program  plans  and  Technical  plans 

•  Changes 

•  Stakeholder  information  (e.g.,  DOTMLPF) 

•  Reference  documents 

•  Policies,  methods,  and  procedures 

•  Technical  Data  Packages 

•  Metrics 

•  Cost  objectives/information 

•  WBS 

•  Schedules 

•  Life  Cycle  Support  Plans 

•  POG’s  (NAVAIR  unique) 

•  Analyses 

•  Reports 

•  Technical  presentations 

•  Requirements 

•  Traceability  matrix 

•  Trade  studies 

•  Functional  and  physical  baselines 

•  Certifications 

•  Specifications 


Sub-process  12:  Outcomes  Management 


Purpose 

Manage  the  outcomes  of  the  technical  effort. 

Tasks 

a)  Capture  the  outcomes,  descriptions  of  methods  and  tools  used, 
decisions  and  assumptions,  lessons  learned,  and  other  data 
that  allow  for  tracking  requirements. 

b)  Perform  configuration  management. 

c)  Perform  change  management. 

d)  Perform  interface  management. 

e)  Perform  risk  management. 

f)  Perform  data  and  document  management. 

g)  Manage  the  information  database. 

h)  Manage  and  track  stakeholder  requirements,  system  technical 
requirements,  logical  solution  representations,  physical  solution 
representations,  derived  technical  requirements,  specified 
requirements,  approved  changes,  and  validation  results. 


Agents 

PM 


Tools 

Rqmts  Mgmt  &  Sys  Arch  Db 


Systems  Engineering 


Next  Process 

Control  Process 
Sub-process  13 


Outputs 

•  Program  Information  (SP  13) 


Exit  Criteria 

Outputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents 


SEP 


Deficiencies  and  discrepancies 

Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 

References 

Standard  References 


Metrics  and  Measures 

Information  is  accurate  and  available  in  a  timely  manner  as 
defined  by  the  program. 
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Sub-process  13:  Information  Dissemination 


Preceding  Process 

Information  requests  could 
come  from  any  of  the  other  32 
sub-processes 


Purpose 

Ensure  that  required  and  requested  information  is  disseminated  in 
accordance  with  the  agreement,  project  plans,  enterprise  policies, 
and  enterprise  procedures. 


Next  Process 

Sub-process(es)  corresponding 
to  the  requested  infomation 


Inputs 

•  Program  Information  (SP  12) 

•  Requests  for  information  (SP  All) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 


Tasks 


a)  Provide  technical  progress  status. 

b)  Provide  technical  planning  information. 

c)  Disseminate  approved  and  controlled  requirements. 

d)  Provide  information  for,  and  from,  technical  reviews. 

e)  Make  available  design  data  and  schema. 

f)  Make  available  lesson  learned. 

g)  Report  variances. 

h)  Disseminate  data  deliverables. 

i)  Disseminate  approved  changes. 

j)  Disseminate  directives. 


Agents 

Information  Specialist 
Data  and  Document  Manager 
Systems  Engineering 

Acquirer 

Supplier 


Tools 

Microsoft  Word 
Excel  Spreadsheet 
Enterprise  Data  Repository 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Program  information  (SP  requesting  sub¬ 
process)  (SP  ALL) 

•  Completed  request  for  information  forms 
(SP  12) 

•  Status  of  Information  dissemination  (SP  12) 


Exit  Criteria 

Outputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents. 


References 

Standard  References 

Security  control  directives  for  the  handling,  packaging  and 
transmittal  of  classified  information 
CMMI  2001 


Metrics  and  Measures 

Percent  of  on-time  deliveries  of  information  requested. 

Percent  of  on-time  deliveries  of  information  required. 

Number  of  complaints  on  the  quality  of  disseminated  information. 
Number  of  security  violations  for  improper  handling,  storage,  and 
transmittal  of  classified  materials. 
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Preceding  Process  __ 

Systems  Analysis  Process 
Sub-process  22 

Requirements  Validation  Process 
Sub-process  26 


Sub-process  14:  Acquirer  Requirements 


Inputs 

•  ICD  -  formerly  MNS  (User,  Fleet) 
(EXT) 

•  CDD  -  formerly  ORD  (OPNAV)  (EXT) 

•  Engineering  Investigation  Reports 
(User,  Fleet)  (EXT) 

•  Utilization  and  Readiness  Reports 
(NALCOMIS)  (EXT) 

•  Specifications  from  higher  level 
system  building  blocks  (EXT) 

•  Sponsor  High-Level  Operational. 
Concept  Graphic  (OV-1 )  arch.  (EXT) 

•  Effectiveness  Analysis  reports  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Acquirer  Requirements  Validation 
Revisions  (SP  26) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Purpose 

Define  a  validated  set  of  acquirer  requirements  for  the  system,  or 
portion  thereof. 

Tasks 

a)  Identify,  collect,  and  prioritize  assigned,  customer,  user,  or 
operator  requirements  for  the  system,  or  portion  thereof, 
including  any  requirements  for  development,  production,  test, 
deployment/installation,  training,  operations, 
support/maintenance,  and  disposal  of  the  system’s  products. 

b)  Ensure  that  the  resulting  set  of  requirements  agrees  with  the 
acquirer  needs  and  expectations 

c)  Record  the  resulting  set  of  acquirer  requirements  in  the 
established  information  database. 


Agents 

Tools 

Acquirer  and  User 

Survey 

Concepts  Analysis 

Questionnaire 

Cost  Analysis 

QFD  Capture 

Fleet  Project  Team  (FPT) 

M&S 

Ops  Analysis 

Queuing  Methodology 

R&M 

IDEF 

Systems  Engineering 

Rqmts  Mgmt  &  Sys  Arch  Db 

Next  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 

Sub-process  4,  5,  7 
Assessment  Process 
Sub-process  10,  11 
Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  16 
Systems  Analysis  Process 
Sub-process  22 

Requirements  Validation  Process 
Sub-process  26 
System  Verification  Process 
Sub-process  31 

End  Products  Validation  Process 
Sub-process  33 _ 


Outputs 

All  outputs  should  be  archived  SP  12 

•  ICD  -  formerly  MNS  (SP  2,  4,  7,  1 0,  1 1 ,  1 6, 

31. 33) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  MOE(SP5,  7,  16) 

•  CDD  or  CPD  -  formerly  ORD  (SP  2,  4,  7, 

10.11.16.31.33) 

•  Specifications  from  higher  level  system 
building  blocks  (SP  16) 

•  Acquirer  Requirements  (SP  5,  16,  26) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 

Systems  Engineering  &  Analysis  (Blanchard) 

MIL-STD-498 

CMMI  2001 


Metrics  and  Measures 

Percent  completion  of  analysis  and  output  products. 
Percent  of  acquirer  requirements  that  have  been  validated. 
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Preceding  Process  _ 

Planning  Process 
Sub-process  4,  8 
Systems  Analysis  Process 
Sub-process  22 

Requirements  Validation  Process 
Sub-process  27 


Sub-process  15:  Other  Stakeholder  Requirements 


Inputs 

•  List  of  stakeholders  and  roles  (SP  4) 

•  TWP  (SP  8) 

•  SOO  (SP  8) 

•  SOW  (SP  8) 

•  Effectiveness  Analysis  Reports  (SP 
22) 

•  Effectiveness  Models  (SP  22) 

•  Other  stakeholder  requirements 
validation  revisions  (SP  27) 

•  DoD/Naval  policy  and  directives  (EXT) 

•  Federal/International  Laws  and 
regulation  (EXT) 

•  International/  National  standards 
(EXT) 

•  Team  /  Project  objectives,  constraints, 
and  policy  (EXT) 


Entry  Criteria 

Inputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents 


Purpose 

Define  a  validated  set  of  other  stakeholder  requirements  for  the 
system,  or  portion  thereof. 

Tasks 

a)  Identify  and  collect  other  stakeholder  requirements  that  can 
constrain  the  system’s  end  products. 

b)  Identify  and  collect  other  stakeholder  requirements  that  can 
constrain  development,  production,  test, 
deployment/installation,  training,  support/maintenance,  and 
disposal  of  the  system  products. 

c)  Identify  and  collect  other  stakeholder  constraints  such  as 
applicable  laws  and  regulations;  technology  base;  standards 
and  specifications;  competitor’s  product  capabilities  and 
trends;  and  interfaces  with  other  evolving  systems  or 
platforms. 

d)  Ensure  that  the  resulting  set  of  requirements  agrees  with  other 
stakeholder  needs  and  expectations 

e)  Record  the  resulting  set  of  stakeholder  requirements  in  the 
established  information  database. 


Agents 

Tools 

Systems  Engineering 

Surveys 

Enterprise  Management 

Questionnaire 

PM,  PEO 

Rqmts  Mgmt  &  Sys  Arch  Db 

Logistics,  Manufacturing 

Depot 

T&E 

Other  Syscoms 

Next  Process 

Planning  Process 
Sub-process  5 
Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  16 
Systems  Analysis  Process 
Sub-process  22 

Requirements  Validation  Process 
Sub-Drocess  27 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Other  stakeholder  requirements  (SP  5, 
16,27) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 
IPPD  Handbook 


Metrics  and  Measures 

Percent  completion  of  analysis  and  output  products. 
Percent  of  other  stakeholder  requirements  that  have  been 
validated. 
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Preceding  Process 

Planning  Process 
Sub-process  5 

Requirements  Definition  Process 
Sub-process  14,  15 
Systems  Analysis  Process 
Sub-process  22,  23 
Requirements  Validation  Process 
Sub-process  25,  28 


Inputs 

•  Specifications  from  higher  level  system 
building  blocks  (SP  14) 

•  ICD  -  formerly  MNS  (SP  14) 

•  CDD  -  formerly  ORD  (SP  14) 

•  Sponsor  High-Level  Operational. 
Concept  Graphic  (OV-1)  arch.  (EXT) 

•  MOE  (SP  14) 

•  Acquirer  requirements  (SP  14) 

•  Other  stakeholder  requirements  (SP  15) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Trade-off  Analysis  Technical  Report  (SP 
23) 

•  Requirement  statements  validation 
revisions  (SP  25) 

•  System  technical  requirements  validation 
revisions  (SP  28) 

•  TDP  (SP  5) 

•  Technology  Roadmap  (SP  5) 

•  Life  Cycle  Support  Plans  (SP  5) 

•  P3I  (SP  5) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


Sub-process  16:  System  Technical  Requirements 


Purpose 

Define  a  validated  set  of  system  technical  requirements. 

Tasks 

a)  Establish  required  transformation  rules,  priorities,  inputs, 
outputs,  states,  modes,  and  configurations 

b)  Define  operational  requirements. 

c)  Define  performance  requirements. 

d)  Analyze  acquirer  and  other  stakeholder  requirements,  and 
derived  functional  and  performance  requirements. 

e)  Identify  and  resolve  requirements  that  have  questionable 
utility  or  have  unacceptable  risk  of  not  being  satisfied. 

f)  Resolve  identified  conflicts  between  the  requirements. 

g)  Prepare  a  set  of  system  technical  requirement  statements  that 
are  well  formulated. 

h)  Ensure  that  the  set  of  system  technical  requirements  is 
correct. 

i)  Record  the  resulting  set  of  system  technical  requirements  in 
the  established  information  database. 


Agents 

Logistics 
Ops  Analysis 
Systems  Engineering 
Test 

Specialty  Engineering 
User 


Tools 

FFBD 

QFD 

Context  Diagram 
Timeline  Analysis 


Next  Process 

Planning  Process 

Sub-process  5,  6,  8 
Assessment  Process 
Sub-process  10,  1 1 
Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  17,  18 
Systems  Analysis  Process 
Sub-process  22,  23 
Requirements  Validation  Process 
Sub-process  25,  28 
System  Verification  Process 
Sub-process  30 


Outputs 

All  outputs  should  be  archived  SP  12 

Utilization  environment  (SP  16) 

Verification  approach  (SP  16) 

Operational  Profiles  (SP  16,  17) 

Physical  and  functional  requirements  (SP  16,  17) 
Mission  Profiles  (SP  16,  17) 

Cycle  timelines  (SP  16,  17) 

MOP  (SP  16,  17) 

KPP  (SP  10,  16,  17) 

Functional  performance  (SP  16,  17) 

Human  interface  requirements  (SP  16,  17) 

Function  concurrency  /  capacity  (SP  16,  17) 
Technology  constraints  (SP  16,  18) 

Design  constraints  (SP  16,  18) 

Enabling  products  requirements  (SP  16,  17) 
Conflicting  requirements  (SP  16,  17) 

Effectiveness  Analysis  Request  (SP  22) 

Trade  Options  and  Constraints  (SP  23) 

System  Requirements  Document  (SRD)  (SP  11,  17) 
System  Technical  Requirements  (SP  5,  6,  8,  11,  17, 
28,  30) 


Exit  Criteria 

Outputs  have  been  reviewed  and  approved  by  the 
appropriate  agents 


References 

Standard  References 
IEEE  1220 
MIL-STD  499B 

System/Subsystem  Specification  (SSS)  Data  Item  Description  (DI-IPSC-81431) 
World  Class  Example,  Jerry  Lake,  1999. 


Metrics  and  Measures 

Percent  completion  of  analysis  and  output  products 

Percent  of  system  technical  requirements  that  have  been  validated 


25, 
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Preceding  Process 

Requirements  Definition  Process 
Sub-process  16  ^ 

Systems  Analysis  Process  ” 

Sub-process  22,  23 
Requirements  Validation  Process 
Sub-process  25 
Sub-process  29 


Inputs 

•  System  Technical  Requirements  (SP  16) 

•  Operational  Profiles  (SP  16) 

•  Physical  and  functional  requirements  (SP 
16) 

•  Mission  Profiles  (SP  16) 

•  Cycle  timelines  (SP  16) 

•  MOP  (SP  16) 

•  KPP  (SP  16) 

•  Functional  performance  (SP  16) 

•  Human  interface  requirements  (SP  16) 

•  Function  concurrency  /  capacity  (SP  16) 

•  Enabling  products  requirements  (SP  16) 

•  Conflicting  requirements  (SP  16) 

•  SRD  (SP  16) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Trade-off  Analysis  Technical  Report  (SP 
23) 

•  Requirement  statements  validation 
revisions  (SP  25) 

•  Logical  solution  representation  validation 
revisions  (SP  29) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


Sub-process  17:  Logical  Solution  Representations 
(Functional  Analysis) 

Purpose 

Define  one  or  more  validated  sets  of  logical  solution  representations  that 
conform  with  the  technical  requirements  of  the  system. 


Tasks 


a) 


b) 

c) 

d) 

e) 

f) 


Select  and  implement  one  or  more  appropriate  approaches  to 
providing  an  abstract  definition  of  the  solution  to  the  system  technical 
requirements. 

Establish  sets  of  logical  solution  representations 
Assign  (i.e.,  perform  Requirements  Allocation  of)  system  technical 
requirements  to  elements  of  the  logical  solution  representations. 
Identify  and  define  derived  technical  requirement  statements 
resulting  from  tasks  a)  and  b). 

Ensure  that  each  set  of  logical  solution  representations  is  correct. 
Record  the  resulting  sets  of  logical  solution  representations,  the  set 
of  derived  technical  requirement  statements,  and  any  unassigned 
system  technical  requirements,  along  with  source  rationale  and 
assumptions  in  the  established  information  database. 


Agents 

Tools 

Systems  Engineering 

Rqmts  Mgmt  &  Sys  Arch  Db 

R&M 

Human  Systems  Integration 

Safety 

Design 

Logistics 

Test 

Software  Develooment 

Next  Process 

Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  18 
Systems  Analysis  Process 
Sub-process  22,  23 
Requirements  Validation  Process 
Sub-process  25,  29 


Outputs 

All  outputs  should  be  archived  SP12 

•  Functional  Analysis  Products  (SP  18) 

(FFBD  /  Functional  Decomposition,  Timeline) 

•  Structured  Analysis  Products  (SP  18) 
(Context  /  QFD  /  Data  Dictionaries  /  Entity- 
Relationship  /  M&S  Diagrams) 

•  Object  Oriented  Analysis  Products  (SP  18) 
(Classical  /  Behavior  /  Domain  /  Use  Case 
Analyses) 

•  N2  /  FMEA  /  FMECA  /  RAS  (SP  1 8) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Trade  options  and  constraints  (SP  23) 

•  Derived  Technical  Requirements  (SP  25) 

•  Logical  Solution  Representation  (SP  18,  29) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 

Systems  Engineering  Analysis  (Blanchard) 

Standard  Practice  for  Performing  FMECA  (MIL-STD-1629), 
DI-ILSS-81 163A,  DI-GDRO-81222 
Object  Oriented  Analysis  &  Design  (Booch) 

Modern  Structured  Analysis  (Yourdon) 


Metrics  and  Measures 

Percent  Completion  of  Logical  Solution  Products 
Percent  of  Logical  Solution  Products  that  have  been 
validated. 
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Preceding  Process 

Requirements  Definition  Process 
Sub-process  16 
Solution  Definition  Process 
Sub-process  17 
Systems  Analysis  Process 
Sub-process  22,  23,  24 
Requirements  Validation  Process 
Sub-process  25 


Inputs 

•  Design  constraints  (SP  16) 

•  Technology  constraints  (SP  16) 

•  Functional  Analysis  Products  (SP  17) 
(FFBD  /  Functional  Decomposition, 
Timeline) 

•  Structured  Analysis  Products  (SP  17) 
(Context  /QFD  /  Data  Dictionaries  /  Entity- 
Relationship  /  M&S  Diagrams) 

•  Object  Oriented  Analysis  Products  (SP 
17) 

(Classical  /  Behavior  /  Domain  /  Use  Case 
Analyses) 

•  N2  /  FMEA  /  FMECA  /  RAS  (SP  1 7) 

•  Logical  Solution  Representation  (SP  17) 

•  Effectiveness  Analysis  Report  (SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Trade-off  Analysis  Technical  Report  (SP 
23) 

•  Risk  Analysis  Report  (SP  24) 

•  Requirement  statements  validation 
revisions  (SP  25) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


Sub-process  18:  Physical  Solution  Representation 

(Synthesis) 


Purpose 

Define  a  preferred  set  of  physical  solution  representations  that  agrees  with 
the  assigned  logical  solution  representations,  derived  technical 
requirements,  and  system  technical  requirements. 


Tasks 

a) 


b) 


c) 


d) 


f) 

g) 


Analyze  logical  solution  representations,  derived  technical 
requirements,  and  any  unassigned  system  technical  requirements. 
Assign  representations  from  Sub-process  17,  unassigned  system 
technical  requirements,  and  derived  technical  requirements  to  physical 
entities  that  will  make  up  a  physical  solution. 

Generate  alternative  physical  solutions  by: 

1)  Identification  and  definition  of  physical  interfaces 

2)  Identification  and  analysis  of  critical  parameters 

3)  Identification  and  assessment  of  physical  solution  options 

4)  Performance  of  system  analysis 

Identify  and  define  derived  technical  requirement  statements  resulting 
from  tasks  a),  b),  and  c)  that  are  stated  acceptably  in  accordance  with 
Sub-process  25. 

Select  the  preferred  physical  solution  representation  for  further 
characterization  into  a  design  solution. 

Ensure  that  the  selected  physical  solution  representation  is  consistent. 
Record  the  selected  physical  solution  representation. 


Aqents 

Tools 

Systems  Engineering 

Rqmts  Mgmt  &  Sys  Arch  Db 

R&M 

SBD 

Human  Systems  Integration 

N2  Diagrams 

Safety 

RAS 

Security 

Concept  DescriptionSheets 

Design,  Logistics,  Test, 

Producibility,  Software  Design 

Design  Sheets 

Next  Process 

Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  19 
Systems  Analysis  Process 
Sub-process  22,  23,  24 
Requirements  Validation  Process 
Sub-process  25 
System  Verification  Process 
Sub-process  30 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Trade  options  and  constraints  (SP  23) 

•  Risk  Analysis  Request  (SP  24) 

•  Physical  Solution  Options  (SP  18) 

•  Derived  Technical  Requirements  (SP  25) 

•  Selected  physical  solution  representation 
(SP  19,  30) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 

Systems  Engineering  Analysis  (Blanchard) 

Standard  Practice  for  Performing  FMECA  (MIL-STD-1629) 


Metrics  and  Measures 

Percent  Completion  of  Physical  Solution  Products 
Percent  of  Physical  Solution  Products  that  have 
been  validated. 
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Preceding  Process 

Assessment  Process 
Sub-process  10 

Solution  Definition  Process  | 
Sub-process  18 

Requirements  Validation  Process 
Sub-process  25 
System  Verification  Process 
Sub-process  30,  31 
End  Products  Validation  Process 
Sub-process  33 


Inputs 

•  Deficiencies  and  discrepancies 
(SP10) 

•  Selected  physical  solution 
representation  (SP  18) 

•  Requirement  statements 
validation  revisions  (SP  25) 

•  Design  solution  deficiency  and 
discrepancy  reports  (SP  30) 

•  End  Product  deficiency  and 
discrepancy  reports  (SP  31) 

•  OT/FOT&E  Report  (SP  33) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Sub-process  19:  Specified  Requirements 
(Document  the  Design  Solution) 


Purpose 

Specify  requirements  for  the  design  solution. 


Tasks 


a) 

b) 

c) 


d) 

e) 


Fully  characterize  the  design  solution. 

Ensure  that  the  design  solution  is  consistent  with  its  source 
requirements. 

Specify  requirements  for  the  system,  system  end  products, 
and  subsystems  of  each  end  product,  as  applicable  to  the 
engineering  life-cycle  phase. 

Record  the  design  solution  work  products  in  the  established 
information  database. 

Establish  projects  to  develop  enabling  products  and  to  procure 
those  that  are  off-the-shelf  or  will  be  reused. 


Agents 

Tools 

Systems  Engineering 

Rqmts  Mgmt  &  Sys  Arch  Db 

R&M 

RAS 

Human  Systems  Integration 

Specification  Standards 

Safety 

Design 

Logistics 

Test 

Software  Development 

Next  Process 

Acquisition  Process 
Sub-process  2,  3 
Assessment  Process 
Sub-process  1 1 
Control  Process 
Sub-process  12 
Implementation  Process 
Sub-process  20 
Transition  to  Use  Process 
Sub-process  21 
Requirements  Validation 
Process 

Sub-process  25 
System  Verification  Process 
Sub-process  30,  31,  32 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Specified  Requirements  (SP  2,  3,  11, 20, 
21,25,30,31,32) 

•  Specified  Requirements  Products  (SP  19) 

•  Enabling  products  development  projects 
(SP  32) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 
Systems  Engineering  Analysis 
(Blanchard) 

Standard  Practice  for  Defense 
Specifications  (MIL-STD-100G) 

Data  Item  Descriptions: 

System  /  Subsystem  Specification 
(SSS)  (DI-IPSC-81431) 

Interface  Requirements  Specification 
(IRS)  (DI-IPSC-81434) 

System  Architecture  Design  (SSDD) 
(DI-IPSC-81432) 


Data  Item  Descriptions,  continued: 

Software  Requirements  Specification 
(SRS)  (DI-IPSC-81433) 

Software  Design  Description  (SDD) 
(DI-IPSC-81435) 

Database  Design  Description  (DBDD) 
(DI-IPSC-81437) 

Interface  Design  Description  (IDD) 
(DI-IPSC-81436) 

Software  Product  Specification  (SPS) 
(DI-IPSC-81441) 

User  Software  Version  Description 
(SVD)  (DI-IPSC-81442) 


Metrics  and  Measures 

Percent  Completion  of  Specified  Requirements  Products 
Percent  of  Specified  Requirements  Products  that  have  been 
validated. 


235 


Preceding  Process 

Supply  Process 

Sub-process  1  — 

Planning  Process 
Sub-process  7 
Solution  Definition  Process 
Sub-process  19 

End  Products  Validation  Process 
Sub-process  33 _ 


Inputs 

•  End  Products  (SP  1 ) 

•  Enabling  Products  (SP  1) 

•  Manufacturing  Plans  (SP  7) 

•  QA  Program  Plans  (SP  7) 

•  Specified  Requirements  (SP  19) 

•  OT/FOT&E  Report  (SP  33) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Sub-process  20:  Implementation 


Purpose 

Implement  (build/assemble/code/test)  the  design  (preliminary  or  final) 
in  accordance  with  the  specified  requirements  to  obtain  a  verified  end 
product. 

Tasks 

a)  Receive  from  suppliers,  reuse  from  off-the-shelf  supply,  or  receive 
from  the  acquirer  the  subsystem  products  that  make  up  the  system’s 
end  products,  or,  as  appropriate,  code  or  build  the  end  products. 

b)  Validate  the  subsystem  products  received  or  reused  against  their 
acquirer  requirements. 

c)  Assemble  the  validated  subsystem  products,  or  physically  integrate 
such  products  into  the  respective  test  article  or  end  product  to  be 
verified. 

d)  Verify  each  test  article  or  end  product  against  its  specified 
requirements. 

e)  Ensure  that  the  enabling  products  for  each  associated  process  will 
be  ready  and  available  to  perform  their  intended  support  functions 
required  by  the  system’s  end  products. 

Validate  the  verified  end  products  against  their  acquirer 


requirements. 

Agents 

Tools 

Prime  Contractor  &  Suppliers, 

Design  tools 

Program  Management,  Systems 

Databases 

Engineering,  Mfg,  QA,  Logistics, 

Manufacturing  Tooling 

Testing,  Financial  Management, 

TPM  Tracking  tools/Schedules 

Procurement,  Parts  Management, 

Test  Equipment/Requirements/Analysis 

End  User,  DCMA 

Statistical  Process  Control 

Inspections/Reviews 

Next  Process 

Control  Process 
Sub-process  12 
Transition  To  Use  Process 
Sub-process  21 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Assembled  End  Product(s)  or  Enabling 
Product(s)  (SP  20) 

•  Manufacturing  Process  &  Personnel 
System  (SP  21) 

•  Verified  &  Validated  Integrated  End 
Product  or  Enabling  Product  Report 
(SP  21) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 
FAR  Parts  46  and  52.246 
DFAR  Part  46 
ISO  9001 
MIL-STD-1528A 
MIL-STD-1521B 


DoD-STD-2168 
DoD-STD-2167A 
Defense  Manufacturing  Guide 
Technical  Reviews 
Manufacturing  Management 
Program 


Metrics  and  Measures 

Adherence  to  Schedule  and  Progress  Versus  Plan 
Requirement  Execution  Time  and  Cost 
System  Definition  Detail 

Technical  Performance  Measurement  Resolution 

•  Availability 

•  Reliability 

•  Capability 

•  Effectiveness 
Process  Control  Matrices 
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Preceding  Process 

Acquisition  Process 
Sub-process  2 
Planning  Process 
Sub-process  6 

Solution  Definition  Process 
Sub-process  19 
Implementation  Process 
Sub-process  20 
System  Verification  Process 
Sub-process  32 


Sub-process  21:  Transition  to  Use 


Inputs 

•  Verified  and  Validated 
Integrated  End  Product  or 
Enabling  Product  Report  (SP 
20) 

•  Manufacturing  Process  & 
Personnel  System  (SP  20) 

.  IMS  (SP  6) 


Purpose 

Transition  verified  products  to  the  acquirer  of  the  products  in 
accordance  with  the  agreement. 

Tasks 

Acquire  and  put  in  place  appropriate  enabling  products  to 
carry  out  relevant  transition  to  use  requirements. 

Prepare  end  products  for  shipping  and  storage. 

Store  end  products  awaiting  shipping,  and  ship  or  transport  to 
the  acquirer  the  intended  usage  sites. 

Prepare  sites  where  end  products  will  be  stored,  installed, 
used  or  maintained,  or  serviced. 

Install  end  products  at  the  appropriate  sites. 

Perform  commissioning  to  bring  delivered  or  installed  end 
products  to  operational  readiness  with  appropriate 
acceptance  and  certification  tests  completed. 

Provide  a  parallel  operation  (ghosting)  of  the  new  and  the 
legacy  end  products  so  that  service  is  continuous  during  the 


a) 

b) 

c) 

d) 

e) 

f) 


g) 


•  Specified  Requirements  (SP  19) 

•  Enabling  Product  Readiness 
determination  (SP  32) 

•  ILS  Certification  (SP  2) 

•  Signed  DD  Form  250  (SP  2) 

transition  period. 

h)  Provide  training  for  users,  maintenance,  and  other  personnel. 

i)  Provide  in-service  support 

j)  Deliver  all  planned  support  elements. 

Agents 

Logistics 

FST 

In-Service  Support 

PM 

Tools 

Not  Applicable 

Entrv  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 

References 

Metrics  and  Measures 

Standard  References 

Percentage  of  damaged  products 
On-time  delivery 

Next  Process 

Control  Process 
Sub-process  12 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Operational  system  products  (EXT) 


Exit  Criteria 

Outputs  have  been  reviewed 
and  approved  by  the 
appropriate  agents 
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Preceding  Process  . 

Planning  Process 
Sub-process  5,  7 
Requirements  Definition  Process 
Sub-process  14,  15,  16 
Solution  Definition  Process 
Sub-process  17,  18 
Systems  Analysis  Process 
Sub-process  23 


Sub-process  22:  Effectiveness  Analysis 


Inputs 

•  TPM  (SP  5) 

•  CAIV  decision  criteria  (SP  5) 

•  SEPorSDP(SP7) 

•  Effectiveness  Analysis  Request 
(SP  14,  15,  16,  17,  18,  23) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Purpose 

Perform  effectiveness  analyses  to  provide  a  quantitative  basis  for 
decision-making. 


Tasks 

a) 


b) 

c) 

d) 

e) 


f) 


Plan  effectiveness  analyses  to  include  purpose,  opjectives, 
execution  and  data  collection,  schedule  of  tasks,  resource 
need  and  availability,  and  expected  outcomes. 

Analyze  each  alternative  for  system  and  cost  effectiveness. 
Analyze  each  alternative  for  total  ownership  cost  to  the 
enterprise  and  to  the  acquirer. 

Analyze  the  environmental  impact  of  each  alternative. 
Analyze  each  alternative  for  each  required  operational  profile 
to  provide  an  analytical  confirmation  that  the  alternative 
satisfies  appropriate  requirements. 

Record  effective  analysis  outcomes  in  the  established 
enterprise  data  repository. 


Agents 

Tools 

Systems  Engineering 

Functional,  physical,  and  analytic 

Legal 

models 

Simulation  models 

Virtual  reality  models 

Mathematical  models 

Next  Process 

Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  14,  15,  16 
Solution  Definition  Process 
Sub-process  17,  18 
Systems  Analysis  Process 
Sub-process  23,  24 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Effectiveness  Analysis  Report  (SP  14,  15,  16,  17, 
18,  23,  24) 

•  Effectiveness  Models  (SP  14,  15,  16,  17,  18,  23) 

•  Effectiveness  Analysis  Plan  (SP  22) 

•  Production  Engineering  Assessment  (SP  22) 

•  Test  &  Evaluation  Assessment  (SP  22) 

•  Deployment  &  Installation  Assessment  (SP  22) 

•  Operations  Assessment  (SP  22) 

•  Support  Assessment  (SP  22) 

•  Training  Assessment  (SP  22) 

•  Total  Ownership  Cost  Assessment  (SP  22) 

•  Environmental  Assessment  (SP  22) 

•  Disposal  Assessment  (SP  22) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 

Engineering  Complex  Systems  with  Models  and  Objects,  Oliver, 
etal,  1997 
MIL-STD-499B 

Models  and  Simulations,  DSMC,  Sept  1994 

Systems  Engineering  Management,  Lacy,  1992 

Systems  Engineering  Guidebook,  Martin,  1996 

System  Engineering  Planning  &  Enterprise  Identity,  Grady,  1994 

Virtual  Prototyping,  DSMC,  1993 


Metrics  and  Measures 

Technical  Performance  Measurement  (TPM),  MOE 

Capability,  dependability,  suitability,  and  cost  effectiveness 

Costs  (Unit,  system,  procurement,  acquisition,  life  cycle,  total  ownership) 

CER 
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Preceding  Process 

Assessment  Process  — 

Sub-process  9,  10 
Requirements  Definition  Process 
Sub-process  16 
Solution  Definition  Process 
Sub-process  17,  18 
Systems  Analysis  Process 
Sub-process  22,  24 


Sub-process  23:  Trade-off  Analysis 


Inputs 

•  Trade  Options  and  Constraints 
(SP  16,  17,  18) 

•  Plans  and  schedules  trend 
analysis  (SP  9) 

•  Requirements  trend  analysis 
(SP10) 

•  Effectiveness  Analysis  Report 
(SP  22) 

•  Effectiveness  Models  (SP  22) 

•  Risk  Analysis  Report  (SP  24) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 

Trade-off  problem  definition  is 
complete 


Purpose 

Perform  Trade-off  analyses  to  provide  decision-makers  (i.e., 

Program  Managers  and  Engineers)  with  recommendations, 
predictions  of  the  results  of  alternative  decisions,  and  other 
appropriate  information  to  allow  selection  of  the  best  course  of 
action. 

Tasks 

a)  Plan  Trade-off  analyses  and  develop  a  Trade-off  Plan  of  Action 
and  Milestones  (POA&M). 

b)  Perform  the  Trade-off  analysis  according  to  the  POA&M,  and 
Produce  a  Trade-off  Analysis  Document  and  Trade-off  Study 
Brief. 

c)  Record  the  outcomes  of  the  T rade-off  analysis  in  the  enterprise 
data  repository,  including  assumptions,  details  of  the  analysis, 
lessons  learned,  models  used,  rationale  for  decisions  made, 
recommendations  and  effects,  and  other  pertinent  information 
affecting  the  interpretation  of  the  decision  made. 


Aaents 

Tools 

Program  Management 

Analysis: 

Planning/Documentation: 

System  Engineering 

•  Excel  with  VBA 

•  Project 

Analysis 

•  Visual  Basic,  C 

•  Schedule 

•  Access 

•  Word 

•  Warfare  &  System/ 
Subsystem  Model 

•  PowerPoint 

•  Integrated  Architecture  Products 

Next  Process 

Assessment  Process 
Sub-process  9 
Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  16 
Solution  Definition  Process 
Sub-process  17,  18 
Systems  Analysis  Process 
Sub-process  22,  24 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Trade-off  Analysis  POA&M  (SP  23) 

•  Effectiveness  Analysis  Request  (SP  22) 

•  Risk  Analysis  Request  (SP  24) 

•  Trade-off  Analysis  Technical  Report  (SP 
9,16,17,18) 

•  Trade-off  Analysis  Brief  (EXT) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 

Trade-off  study  is  completei 

Results  are  archived 


References 

Standard  References 

Naval  Operations  Analysis,  Wagner,  et  al,  1999 

Simulation  and  Modeling  Analysis,  Law  and  Kelton,  1 981  &  1982 

System  Engineering  Management,  Lacy,  1992 

AIR  4.10  Warfare  Analysis  Department  Analysis  of  Alternatives’  Process 

AIR  4.10  Warfare  Analysis  Department  ‘Warfare  Analysis’  Process 

AIR  4.10  Warfare  Analysis  Department  ‘Source  Selection  Process’  Process 


Metrics  and  Measures 

Trade-off  study  completion  and  acceptance  by  the 
appropriate  agent 
Adherence  to  schedule 
Adherence  to  funding  plan 
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Preceding  Process 

Planning  Process 
Sub-process  5,  7 
Assessment  Process 
Sub-process  9,  1 0 
Solution  Definition  Process 
Sub-process  18 
Systems  Analysis  Process 
Sub-process  22,  23 


Sub-process  24:  Risk  Analysis 


Inputs 

•  Risk  Management  Strategy  (SP  5) 

•  Risk  Management  Plan  (SP  7) 

•  SEPorSDP(SP7) 

•  Plans  and  schedules  trend  analysis 
(SP  9) 

•  Requirements  trend  analysis  (SP  10) 

•  Risk  Analysis  Request  (SP  18,  23) 

•  Effectiveness  Analysis  Report  (SP  22) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Purpose 

Perform  risk  analyses  to  develop  risk  management  strategies, 
support  management  of  risk,  and  support  decision-making. 

Tasks 

a)  Identification  of  technical  risk,  and  resulting  project  risk,  based 
on  exposure  to  the  probability  of  an  undesireable  consequence 
and  the  effect  of  that  consequence  for  each  Trade-off  analysis 
option  for  each  physical  solution  representation. 

b)  Characterize  risks  by  causes,  possible  effects  or 
consequences,  likelihood  of  occurrence,  options  for  dealing  with 
risks,  how  long  option  is  available,  and  coupling  with  other  risks. 

c)  Prioritize  risks  that  would  likely  cause  harm,  have  the  greatest 
effect  on  the  system,  and  would  require  attention  in  the  near 
term. 

d)  Evaluate  ways  to  avert  risk,  and  determine  the  cost,  schedule, 
and  performance  effects  on  the  project. 

e)  Define  and  implement  a  plan  or  approach  for  averting  each 
significant  risk. 

f)  Record  the  risk  analysis  outcomes  in  the  enterprise  data 
repository  and  communicate  or  use  risk  findings  and  impacts. 


Agents 

Tools 

Typically  a  program  Risk 

Program  Risk  Summaries 

Management  Board  that  is 

(“Rubic’s  cubes”) 

comprised  of  Program 

DSMC  “Weighted  Factors” 

Management  (1.0)  and  Systems 

Schedule  Network  Models 

Engineering 

R&M  Models 

TPM  Tracking  tools 

Integrated  Architecture  Products 

Next  Process 

Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  18 
Systems  Analysis  Process 
Sub-process  23 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Risk  Analysis  Report  (SP  18,  23) 

•  List  of  risks  (SP  24) 

•  Analyses  of  Risk  Severity  (SP  24) 

•  Risk  Summary  Worksheet  (SP  24) 

•  Waterfall  Charts  (SP  24) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 


References 

Standard  References 
DAU  Prog.  Mgr’s  Tool  Kit 
NAVAIRINST  5000.21 
NAVSO  P-3686 
CMMI  2001 
MIL-STD-882 


Metrics  and  Measures 

Qualitative  Risk  Severity 
Quantitative  Risk  Factor 

Analog  of  Nichols  Gowth  Curve  (keeping  up  with  mitigation  plan) 

•  Availability 

•  Reliability 

•  Capability 


240 


Preceding  Process  — 

Planning  Process 
Sub-process  7 

Requirements  Definition  Process 
Sub-process  16 
Solution  Definition  Process 
Sub-process  17,  18,  19 


Inputs 

•  Verification  Plan  (including 
verification  matrix)  (SP  7) 

•  Validation  Plan  (SP  7) 

•  System  Technical  Requirements 
(SP  16) 

•  Derived  Technical  Requirements 
(SP  17,  18) 

•  Specified  Requirements  (SP  19) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Sub-process  25:  Requirements  Statements  Validation  . 

Purpose 

Ensure  that  technical  requirement  statements  and  specified 
requirement  statements,  individually  and  as  sets,  are  well 
formulated.  This  is  validation  of  the  language  of  the  statements 
rather  than  the  content. 

Tasks 

a)  Analyze  each  requirement  statement  from  Sub-processes  16, 

17,  18,  and  19  to  ensure  1)  ability  to  preserve  competitiveness, 
2)  clarity,  3)  correctness,  4)  feasibility,  5)  focus,  6) 
implementability,  7)  modifiability,  8)  removal  of  ambiguity,  9) 
singularity,  1 0)  testability,  1 1 )  verifiability,  and  12)  performance 
based  language  (where  appropriate). 

b)  Analyze  requirement  statements  from  Sub-processes  16,  17, 

18,  and  19  in  pairs  and  sets  to  ensure  1)  absence  of 
redundancy,  2)  connectivity,  and  3)  removal  of  conflicts. 

c)  Record  requirement  statements  validation  outcomes  in  the 
established  enterprise  data  repository. 


Agents 

Systems  Engineering 
Technical  Writer 


Tools 

Rqmts  Mgmt  &  Sys  Arch  Db 


Next  Process 

Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  16 
Solution  Definition  Process 
Sub-process  17,  18,  19 

Outputs 

All  outputs  should  be  archived  SP  12 

•  Requirement  statement  validation 
revisions  (SP  16,  17,  18,  19) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 

(Acceptable  sets  of  requirements 
statements) 


References 

Standard  References 
MIL-STD-961D 

SD-24:  General  Specification  Performance,  Design, 
Characteristics,  and  Construction  of  Aircraft  Weapons  Systems 
Joint  Services  Specification  Guides  (JSSG) 


Metrics  and  Measures 

Percentage  of  validated  requirements  statements 
Percentage  of  requirement  statements  issues 
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Preceding  Process  _i 

Planning  Process  1 

Sub-process  7 

Requirements  Definition  Process 
Sub-process  14 


Inputs 

•  Validation  Plan  (SP  7) 

•  Acquirer  Requirements  (SP  14) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Sub-process  26:  Acquirer  Requirements  Validation 


Purpose 

Ensure  that  the  set  of  defined  acquirer  requirements  agrees  with 
acquirer  needs  and  expectations. 


Tasks 


a) 


b) 


c) 


d) 

e) 


Select  the  methods  and  define  the  procedures  for  validating  that 
the  set  of  acquirer  requirements  from  Sub-process  14  is 
consistent  with  the  level  of  system  structure,  enterprise-based 
life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

Analyze  and  compare  the  identified,  derived  and  collected 
acquirer  requirements  to  the  set  of  defined  acquirer 
requirements,  to  determine  downward  traceability. 

Analyze  and  compare  the  set  of  defined  acquirer  requirements 
to  the  identified,  derived,  and  collected  acquirer  requirements, 
to  determine  upward  traceability. 

Identify  and  resolve  variances,  voids,  and  conflicts  (orphans). 
Record  validation  results  in  the  enterprise  data  repository. 


Agents 

Tools 

Systems  Engineering 

Requirements  Traceability 

Design  Team 

Matrix  Format 

R&M 

Rqmts  Mgmt  &  Sys  Arch  Db 

Safety 

Supportability/Testability 

Next  Process 

Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  14 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Validation  methods  &  procedures 
(SP  26) 

•  Requirements  traceability  matrix 
(SP  26) 

•  Acquirer  requirements  validation  revisions 
(SP  14) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


References 

Standard  References 


Metrics  and  Measures 

Percent  of  Acquirer  Requirements  downward  traceable 

Percent  of  Acquirer  Requirements  upward  traceable 

Percent  of  assumptions  for  Acquirer  Requirements  reviewed  and  approved 

Percent  of  changed  Acquirer  Requirements  revalidated 
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Preceding  Process  __ 

Planning  Process 
Sub-process  7 

Requirements  Definition  Process 
Sub-process  15 


Sub-process  27:  Other  Stakeholder 
Requirements  Validation 


Purpose 

Ensure  that  the  set  of  defined  other  stakeholder  requirements 
agrees  with  other  stakeholder  needs  and  expectations  with  respect 
to  the  system. 


Next  Process 

Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  15 


Inputs 

•  Validation  Plan  (SP  7) 

•  Other  Stakeholder 
Requirements  (SP  15) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Tasks 

a)  Select  the  methods  and  define  the  procedures  for  validating  that 
the  set  of  other  stakeholder  requirements  from  Sub-process  15 
is  consistent  with  the  level  of  system  structure,  enterprise-based 
life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

b)  Analyze  and  compare  the  identified,  derived,  and  collected 
other  stakeholder  requirements  to  the  set  of  defined  other 
stakeholder  requirements,  to  determine  downward  traceability. 

c)  Analyze  and  compare  the  set  of  defined  other  stakeholder 
requirements  to  the  identified,  derived,  and  collected  other 
stakeholder  requirements,  to  deterimine  upward  traceability. 

d)  Identify  and  resolve  variances,  voids,  and  conflicts  (orphans). 

e)  Record  validation  results  in  the  established  enterprise  data 
repository. 


Agents 

Tools 

Systems  Engineering 

Requirements  Traceability  Matrix 

Design  Team 

Format 

R&M 

Rqmts  Mgmt  &  Sys  Arch  Db 

Safety 

Supportability/Testability 

Outputs 

All  outputs  should  be  archived  (SP  12) 

•  Validation  methods  &  procedures  (SP  27) 

•  Requirements  Traceability  Matrix  (SP  27) 

•  Other  stakeholder  requirements  validation 
revisions  (SP  15) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 


Metrics  and  Measures 

Percent  of  Other  Stakeholder  Requirements  downward  traceable 
Percent  of  Other  Stakeholder  Requirements  upward  traceable 
Percent  of  assumptions  for  Other  Stakeholder  Requirements  reviewed 
and  approved 

Percent  of  changed  Other  Stakeholder  Requirements  revalidated 
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Preceding  Process  — 

Planning  Process 
Sub-process  7 

Requirements  Definition  Process 
Sub-process  16 


Inputs 

•  Validation  Plan  (SP  7) 

•  System  Technical 
Requirements  (including 
Design  Information  and 
ICD/CDD  revisions)  (SP  16) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


References 

Standard  References 


Sub-process  28:  System  Technical 
Requirements  Validation 


Purpose 

Ensure  that  the  set  of  defined  system  technical  requirements  agrees  with 
validated  acquirer  and  other  stakeholder  requirements. 

Tasks 

a)  Select  the  methods  and  define  the  procedures  for  validating  that  the 
set  of  system  technical  requirements  from  Sub-process  16  is 
consistent  with  the  level  of  system  structure,  enterprise-based  life-cycle 
phase,  and  Validation  Plan. 

Analyze  and  compare  the  set  of  validated  acquirer  and  other 
stakeholder  requirements  to  the  set  of  defined  system  techncial 
requirements,  to  determine  downward  traceability. 

Analyze  and  compare  the  set  of  defined  system  techncial  requirements 
to  the  validated  set  of  acquirer  and  other  stakeholder  requirements,  to 
determine  upward  traceability. 

Analyze  assumptions  make  with  respect  to  defining  system  technical 
requirements  to  ensure  that  they  are  consistent  with  the  system  being 
engineered. 

Analyze  system  technical  requirements  that  have  been  defined  as 
essential  for  the  design  effort  for  other  life-cycle  considerations  for 
which  there  is  no  parent  requirement  in  the  set  of  acquirer  and  other 
stakeholder  requirements. 

Identify  and  resolve  variances,  voids,  and  conflicts. 

Revalidate  ths  system  technical  requirements  whenever  a  requirement 
change  is  made  that  affects  the  acquirer  requirements,  other 
stakeholder  requirements,  or  system  technical  requirements. 

Record  validation  results  in  the  established  enterprise  data  repository. 


b) 


c) 


d) 


0 

g) 


h) 


Agents 

Systems  Engineering 
Design  Team 
R&M 
Safety 

Su  pporta  bl  ity/T  esta  bi  I  ity 


Tools 

Requirements  Traceability  Matrix 
Format 

Rqmts  Mgmt  &  Sys  Arch  Db 


Next  Process 

Control  Process 
Sub-process  12 

Requirements  Definition  Process 
Sub-process  16 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Validation  Methods  &  Procedures 
(SP  28) 

•  Requirements  Traceability  Matrix 
(SP  28) 

•  System  technical  requirements 
validation  revisions  (SP  16) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Metrics  and  Measures 

Percent  of  System  Technical  Requirements  downward  traceable 
Percent  of  System  Technical  Requirements  upward  traceable 
Percent  of  assumptions  for  System  Technical  Requirements  reviewed 
and  approved 

Percent  of  changed  System  Technical  Requirements  revalidated 
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Preceding  Process 

Planning  Process 
Sub-process  7 
Solution  Definition  Process 
Sub-process  17 


Inputs 

•  Validation  Plan  (SP  7) 

•  Logical  Solution  Representation 
(SP  17) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Sub-process  29:  Logical  Solution  Representations 

Validation 


Purpose 

Ensure  that  the  set  of  logical  solution  representations  agrees  with  the 
appropriately  assigned  subset  of  system  technical  requirements. 

Tasks 

a)  Select  the  methods  and  define  the  procedures  for  validating  that  the  sets 
of  logical  solution  representations  and  derived  technical  requirements 
from  Sub-process  17  are  consistent  with  the  level  of  system  structure, 
enterprise-based  life-cycle  phase,  and  Validation  Plan,  as  appropriate. 

b)  Analyze  and  compare  the  set  of  validated  system  technical  requirements 
to  the  set  of  defined  logical  solution  representations  and  derived  techical 
requirements,  to  determine  downward  traceability. 

c)  Analyze  and  compare  the  set  of  defined  logical  solution  representations, 
derived  technical  requirements,  and  any  unassigned  system  technical 
requirements  to  the  validated  set  of  system  technical  requirements,  to 
determine  upward  treaceability. 

d)  Analyze  assumptions  made  with  respect  to  defining  sets  of  logical 
solution  representations  and  derved  technical  requirements  to  ensure 
that  they  are  consistent  with  the  system  technical  requirements  and  the 
system  being  engineered. 

e)  Identify  and  resolve  variances,  voids,  and  conflicts  (orphans). 

f)  Revalidate  the  sets  of  logical  solution  representations  whenever  a 
requirement  change  is  made  that  affects  the  acquirer  requirements,  other 
stakeholder  requirements,  system  technical  requirements,  or  sets  of 
defined  logical  solution  representations  and  derived  technical 
requirements. 

g)  Record  validation  results  in  the  established  enterprise  data  repository. 


Agents 

Tools 

Systems  Engineering 

Requirements  Traceability 

R&M 

Matrix  Format 

Safety 

Rqmts  Mgmt  &  Sys  Arch  Db 

Supportablity/T  estability 

Next  Process 

Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  17 


Outputs 

•  All  outputs  should  be  archived  SP 

12Validation  Methods  &  Procedures 
(SP  29) 

•  Requirements  Traceability  Matrix 
(SP  29) 

•  Logical  Solution  Representation  validation 
revisions  (SP  17) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


References 

Standard  References 


Metrics  and  Measures 

Percent  of  Logical  Solution  Representation  downward  traceable 
Percent  of  Logical  Solution  Representation  upward  traceable 
Percent  of  assumptions  for  Logical  Solution  reviewed  and  approved 
Percent  of  changed  Logical  Solution  Representation  revalidated 
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Preceding  Process 

Planning  Process 
Sub-process  7,  8 
Requirements  Definition  Process 
Sub-process  16 
Solution  Definition  Process 
Sub-process  18,  19 


Sub-process  30:  Design  Solution  Verification 


Inputs 

•  Verification  Plan  (including  the 
VCRM)  (SP  7) 

•  SEP  or  SDP  (SP  7) 

•  TEMP  (SP  7) 

•  IV&V  Plan  (SP  7) 

•  TWP  (SP  8) 

•  SOO  (SP  8) 

•  SOW  (SP  8) 

•  System  Technical  Requirements 
(SP  16) 

•  Selected  physical  solution 
representation  (SP  1 8) 

•  Specified  Requirements  (SP  19) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


Purpose 

Verify  that  each  end  product  defined  by  the  system  design  solution 
conforms  to  the  requirements  of  the  selected  physical  solution 
representation  for  Hardware  and  Software  (if  applicable). 

Tasks 

a)  Plan  the  design  solution  verification  in  accordance  with  the 
Verification  Plan,  agreement,  applicable  enterprise-based  life- 
cycle  phase,  and  to  the  level  in  the  system  structure. 

b)  Perform  the  planned  design  solution  verification  using  the 
selection  methods  and  procedures  within  the  established 
verification  environment. 

c)  Reverify  according  to  a  redesign  verification  plan,  test  method,  or 
procedure  when  variances  were  determined  to  be  caused  by  poor| 
verification  or  inadequate  verification  environmental  preparation. 

d)  Record  verification  results,  including:  corrective  actions  taken; 
lessons  learned;  outcomes  achieved;  Trade-off,  effectiveness, 
and  risk  analyses  completed  with  resulting  key  decisions;  test 
activities  completed;  variances;  and  the  verified  design  solution  in 
the  established  enterprised  data  repository. 


Agents 

Manufacturing 

Logistics 

Product  Assurance  -  Speciality  Eng. 
Software  Development 
Systems  Engineering 
T  &  E 


Tools 

Modeling  &  Simulation 
Stress  Testing 
Software  Analysis  Tools 
Integrated  Architecture  products 
Rqmt  Management  Tools  Summary 


Next  Process 

Assessment  Process 
Sub-process  10,  11 
Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  19 
System  Verification  Process 
Sub-process  31 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Demonstrated  Test  Readiness  Report 
(DTRR)  (SP  12,31) 

•  Design  solution  verification  report  (SP  31) 

•  Design  Solution  Deficiency 
andDiscrepancy  Reports  (SP  10,  11,  12, 
19,31) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 


References 

Standard  References 

ANSI/EAI  632  (Para.  4.5.2)  Process  for  Engineering  a  System 
TE000-AB-GTP-010  Parts  Derating  Requirements  and  Applications 
Manual  for  Navy  Electronic  Equipment 
Equivalent  to  MIL-STD-2164  Environmental  Stress  Screening  Process  for 
Electronic  Equipment 

Equivalent  to  MIL-STD-454  Standard  General  Requirements  for  Electronic 
Equipment 

DOD  4245. 7-M  Transition  From  Development  to  Production 
NAVSO  P-6071  Best  Practices-The  Transition  from  Development  to 
Production 
CMMI  2001 


Metrics  and  Measures 

Test  Schedules  (including  dates,  milestones,  etc.)  are  met. 
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Preceding  Process 

Supply  Process 
Sub-process  1 
Planning  Process 
Sub-process  7 

Requirements  Definition  Process 
Sub-process  14 
Solution  Definition  Process 
Sub-process  19 
System  Verification  Process 
Sub-process  30 


Sub-process  31:  End  Product  Verification 


Inputs 

•  End  Products  (“as  built”  production 
representative)  (SP  1 ) 

•  Enabling  Products  (SP  1) 

•  TEMP  (SP  7) 

•  Verification  Plan  including  VCRM 
(SP  7) 

•  ICD  -  formerly  MNS  (SP  14) 

•  CDD  or  CPD  -  formerly  ORD  (SP 
14) 

•  Specified  requirements  (SP  19) 

•  DTRR  (SP  30) 

•  Design  solution  verification  report 
(SP  30) 

•  Design  solution  deficiency  and 
discrepancy  report  (SP  30) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agents 
(approved  Test  Plan  including  risk 
mitigation) 


Purpose 

Verify  that  an  end  product  (“as  built”  production  representative)  to 
be  delivered  to  an  acquirer  conforms  to  its  specified  requirements. 

Tasks 

a)  Plan  the  end  product  (system  and  subsystem,  “as  built”)  verification  in 
accordance  with  the  Verification  Plan,  agreement  (normally  associated 
with  detailed  developmental  test  plans),  applicable  enterprise-based 
life  cycle-phase,  and  level  in  the  system  structure. 

b)  Verify  the  end  product  (system/subsystem,  “as  built”),  using  the 
selected  methods  and  procedures  within  the  established  verification 
environment. 

c)  Reverify  according  to  a  redesigned  verification  plan,  test  method,  or 
procedure  when  variances  were  determined  to  be  caused  by  poor 
verification  or  inadequate  verification  environmental  preparation. 

d)  Record  verificaiton  results,  including  corrective  actions  taken;  lessons 
learned;  outcomes  achieved;  Trade-off,  effectiveness,  and  risk 
analyses  completed  with  resulting  key  decisions;  test  activities 
completed;  variances;  and  the  verified  end  products  in  the  established 
enterprise  data  repository. 


Aqents 

Tools 

T&E 

Ranges  (flight  test) 

R&M 

Test  Plans  (System,  Subsystem, 

Systems  Engineering 

and  Integrated) 

Human  Factors 

Facilities/Labs  (ground  tests) 

Acquirer 

Aircraft  and  systems,  and  ALL 

PEO/PM 

supporting  systems  under  test 

Operators  /  Users 

Flight  Clearance 

Developer  /  Contractor 

Deficiency  Database 

Next  Process 

Assessment  Process 
Sub-process  10,  1 1 
Control  Process 
Sub-process  12 
Solution  Definition  Process 
Sub-process  19 

End  Products  Validation  Process 
Sub-process  33 


Outputs 

All  outputs  should  be  archived  SP  12 

•  Detailed  developmental  test  plans  (SP  31) 

•  Developmental  test  methods  (SP  31) 

•  Developmental  test  procedures  (SP  31) 

•  End  Product  Deficiency  and  Discrepancy 
Reports  (SP  10,  11,  19) 

•  DT/OT  Transition  Report  (SP  33) 

•  Report  of  Test  Results  with  limitations  and 
constraints  for  OT  (SP  33) 

•  Operational  Advisory  Document  (SP  33) 


Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate  agents. 
(Completion  of  the  Verification  phase 
evaluated  results  and  reported 
conclusions.) 


References 

Standard  References 

NAVAIR  Test  Plan  Instruction  3960.2  series 

NATOPS  Flight  and  Weapon  Systems  Manual  (for  each  platform) 

Range  Safety  Operation  Guides  (for  each  range  operated  on) 

Test  Squadron  SOP’s/  Facility  SOP’s 

U.S.  Naval  Test  Pilot  School  Flight  Test  Manual 

Software  Requirements  Specifications 

SAR’s/STR’s 

NAVAIR  NTAB  Instruction  3960.5  Manufacturers  Specifications 
CMMI  2001 


Metrics  and  Measures 

Deficiencies  (Part  I,  II,  III)  based  on  verification 

•  Specification  Compliance 

•  TEMP  Compliance 

•  Mission  Relation/Impact 

Earned  Value  Measurements  (cost,  performance,  test  completion, 
ground/lab/flight  hours,  and  data  points) 

Test  Schedule 

End  Products  Deficiency  Reports 
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Preceding  Process 

Supply  Process 
Sub-process  1 
Planning  Process 
Sub-process  5 
Solution  Definition  Process 
Sub-process  19 


Inputs 

•  Enabling  Products  (SP  1) 

•  List  of:  Methods  and  Tools, 
Facilities,  Equipment,  Training 
(SP  5) 

•  Specified  Requirements  (SP  19) 

•  Enabling  Products  Development 
Projects  (SP  1 9) 


Sub-process  32:  Enabling  Products  Readiness 


Purpose 

Determine  the  readiness  of  enabling  products  for  development, 
production,  test,  deployment/installation,  training,  support/ 
maintenance,  and  retirement  or  disposal. 


Tasks 

a)  Plan  enabling  product  readiness  determination  and  associated 
process  proofing. 

b)  Do  the  planned  enabling  product  readiness  determination  and 
associated  process  proofing. 

c)  Reaccomplish  readiness  determination  when  variances  were 
determined  to  be  caused  by  poor  readiness  or  proofing  conduct, 
or  by  indadequate  environmental  preparation. 

d)  Record  readiness  determination  and  process  proofing  results; 
and  the  verified  enabling  products  and  proofing  of  associated 
processes  in  the  established  enterprise  data  repository. 


Next  Process 

Control  Process 
Sub-process  12 
Transition  to  Use  Process 
Sub-process  21 
System  Verification  Process 
Sub-process  32 


Outputs 

All  outputs  should  be  archived  SP12 

•  Enabling  Products  Readiness 
Determination  (SP  12,  21) 

•  Enabling  Products  Readiness 
Assessment  Plan  (SP  32) 


Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


Agents 

Tools 

System  Engineering 

Enterprise  Data  Repository,  Integrated 

Logistics 

Architecture  Products,  Manufacturing 

T&E 

Tooling,  TPM  Tracking  tools/ 

Training 

Manufacturing 

Schedules,  Test  Equipment  & 

Software,  Statistical  Process  Control, 
Manufacturing  Simulations,  CAD/CAM, 

Program  Manager 

Removal  Tools,  Flight  Simulators, 
Training  Manuals,  Readiness  archives 
and  databases 

Exit  Criteria 

Outputs  have  been  reviewed  and 
approved  by  the  appropriate 
agents 


References 

Standard  References 
DOD  5000.2R,  Parts  3.3,  5.2,  &  7.4 
MIL-STD-1521B 
MIL-STD-499B,  Parts  5.5  &  5.7 

NAVSO  P-6071  Best  Practices,  Sections  5.0,  6.0,  7.0,  9.0,  &  10.4 
DOD  4245. 7-M  Transition  from  Development  to  Production  Sections,  4.0, 
5.0,  6.0,  8.0,  &  9.0 
DAU  Program  Manager’s  Tool  Kit 
CMMI  2001 


Metrics  and  Measures 

Adherence  to  Schedule  and  Progress  versus  Plan 
Sub-process  execution  time  and  cost 
System  Definition  Detail 

Technical  Performance  Measurement  Resolution 

•  Availability 

•  Reliability 

•  Capability 

•  Effectiveness 
Process  Control  Matrices 
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Preceding  Process  _| 

Supply  Process 
Sub-process  1 
Planning  Process 
Sub-process  7 

Requirements  Definition  Process 
Sub-process  14 
System  Verification  Process 
Sub-process  31 

Inputs 

•  End  Products  (SP  1) 

•  Enabling  Products  (SP  1) 

•  Validation  Plan  (Operational  Test 
Plan)  (SP  7) 

•  TEMP  (SP  7) 

•  OTRR  (Internal  or  SP  7) 

•  ICD  -  formerly  MNS  (SP  14) 

•  ODD  -  formerly  ORD  (SP  14) 

•  DT/OT  Transition  Report  (SP  31) 

•  Report  of  Test  Results  with 
limitations  and  constraints  for 
(OT)  (SP  31) 

•  Operational  Advisory  Document 
(SP  31) 

Entry  Criteria 

Inputs  have  been  reviewed  and 
approved  by  the  appropriate  agent. 
For  most  programs,  the  appropriate 
Development  Test  (DT)  must  have 
been  successfully  completed  and  a 
DT  report  issued. 


References 

Standard  References 
DRAFT  MIL-STD-499B 
NAVAIR  3960.2  Series 
CMMI  2001 
MIL-STD-3960.2 
IEEE/EIA  12201 


Sub-process  33:  End  Products  Validation 

Purpose 

Ensure  that  an  end  product,  or  an  aggregation  of  end  products, 
conforms  to  its  validated  acquirer  requirements. 

Tasks 

a)  Determine  the  type  of  end  product  validation  required  and  the  exit 
criteria,  including  the  acquirer  requirements  applicable  to  the 
sytem  end  products  being  validated. 

b)  Acquire  the  test  article,  or  aggregation  of  end  products,  for  the 
validation  as  appropriate  to  the  enterprise-based  life-cycle  phase 
and  level  of  system  structure. 

c)  Conduct  the  end  products  validation  in  accordance  with  the 
Validation  Plan,  as  required  in  the  agreement,  to  show 
conformance  with  appropriate  requirements;  collect  and  analyze 
validation  outcomes  to  identify  any  variances;  and  do  appropriate 
process  tasks  to  resolve  variances  and  repeat  appropriate 
verifications  and  validations. 

d)  Revalidate  with  improved  or  corrected  procedures  and  equipment 
when  variances  were  caused  by  poor  test  conduct  and  conditions. 

e)  Record  the  validation  outcomes,  procedures,  assumptions, 
lessons  learned,  and  other  pertinent  information  about  the 
validation  and  results  in  the  established  enterprise  data 
repository,  to  provide  traceability. 

Next  Process 

Acquisition  Process 

Sub-process  2 

Control  Process 

Sub-process  12 

Solution  Definition  Process 
Sub-process  19 
Implementation  Process 
Sub-process  20 

Outputs 

All  outputs  should  be  archived  SI 

•  OTRR  Plan  (SP  33) 

•  OTRR  certification  message  (S 

•  OT/FOT&E  Report  (SP  1 9,  20) 

Agents 

Tools 

Exit  Criteria 

OPTEVFOR 

SIL 

Outputs  have  been  reviewed 

DOT&E 

HIL 

and  approved  by  the 

Systems  Engineering 

M&S 

appropriate  agents. 

T&E 

Flight  Test 

COMOPTEVFOR 

Metrics  and  Measures 

OTRR  is  achieved  within  program  schedule 
Operational  test  procedures  and  processes  are  carried  out 
according  to  the  TEMP 
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Appendix  I  -  Acronyms 


ACAT 

Acquisition  Category 

ACWP 

Actual  Cost  of  Work  Performed 

AIAA 

American  Institute  of  Aeronautics  and  Astronautics 

ANSI 

American  National  Standards  Institute 

AOA 

Analysis  of  Alternatives  (formerly  called  COEA) 

APB 

Acquisition  Program  Baseline 

APMSE 

Assistant  Program  Manager  for  Systems  Engineering 

AVDEP-HDBK 

AWESim™ 

Simulation  software 

BCWP 

Budget  Cost  of  Work  Performed 

BCWS 

Budget  Cost  of  Work  Scheduled 

BES 

Budget  Estimate  Submission 

BPS 

BITs  per  second 

C/S  SR 

Cost/Schedule  Status  Reports 

C4ISR 

Command,  Control,  Communication,  Computers,  Intelligence 

CAD 

Computer  Aided  Design 

CAE 

Component  Acquisition  Executive;  Computer  Aided  Engineering 

CAIV 

Cost  as  an  Independent  Variable 

CAM 

Computer  Aided  Manufacturing 

CBD 

Commerce  Business  Daily 

CDD 

Capability  Development  Document 

CDR 

Critical  Design  Review 

CDRL 

Contract  Data  Requirements  List 

CDS 

Concept  Description  Sheet 

CE 

Concept  Exploration 

CER 

Cost  Estimating  Relationships 

Cl 

Configuration  Item 

CITIS 

Contractor  Integrated  Technical  Information  Services 

CM 

Configuration  Management;  Contract  Management 

CMMI 

Capability  Maturity  Model  Integration 

COCOMO 

Constructive  Cost  Model 

COEA 

Cost  of  Operations  Effectiveness  Analysis  (obsolete  -  see  AOA) 

COMOPTE  VF  OR 

Commander,  Operational  Test  and  Evaluation  Force  (Navy) 

CORE™ 

Requirements  Management  &  System  Architecture  Database  Software 

COTS 

Commercial  Off  The  Shelf 

CPI 

Cost  Performance  Index 

CPM 

Critical  Path  Method 

CPR 

Cost  Performance  Reports 

CRLCMP 

Computer  Resources  Life  Cycle  Management  Plan 

CSCI 

Computer  Software  Configuation  Item  (aka  SI) 

CV 

Cost  variance 

CWBS 

Contract  Work  Breakdown  Structure 
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DAD 

Defense  Acquisition  Deskbook 

dB 

Decibel 

DBDD 

Database  Design  Description 

Deg 

Degree 

DFARs 

Defense  Federal  Acquisition  Regulations 

DID 

Data  Item  Description 

DCMA 

Defense  Contractor  Management  Agency 

DoD 

Department  of  Defense 

DoDSSP 

Department  of  Defense  Single  Stock  Point 

DoN 

Department  of  Navy 

DOORS 

Demonstration  of  Dynamic  Object-Oriented  Requirements  System 

DOT&E 

Director,  Operational  Test  and  Evaluation  (OSD) 

DS 

Design  Sheet 

DSMC 

Defense  Systems  Management  College 

DT 

Developmental  Test 

DTR 

Derived  Technical  Requirements 

DTRR 

Demonstration  Test  Readiness  Report 

EDA 

Electronic  Design  Automation 

EIA 

Electronic  Industries  Alliance 

EMC 

Electromagnetic  Compatibility 

EMD 

Engineering  and  Manufacturing  Development 

replaced  with  System  Development  and  Demonstration  (SDD) 

EMI 

Electromagnetic  Interference 

ENMIG 

Earned  Value  Management  Implementation  Guide 

EVM 

Earned  Value  Management 

EVMS 

Earned  Value  Management  System 

EXT 

External  supplier  or  acquirer,  i.e.,  associated  product  is  external,  unspecified, 
and  neither  an  input  from,  or  an  output  for,  a  sub-process. 

FAR 

Federal  Acquisition  Regulation 

FCA 

Functional  Configuration  Audit 

FFBD 

Functional  Flow  Block  Diagram 

FIS 

Facility  Interface  Sheet 

FMEA 

Failure  Modes  and  Effectiveness  Analysis 

FMECA 

Failure  Modes  and  Effectiveness  Critical  Analysis 

FOT&E 

Follow-On  Test  &  Evaluation 

FPT 

Fleet  Project  Team 

FST 

Fleet  Support  Team 

Ft 

Feet 

FTA 

Fault-tree  analysis 

FTEG 

Flight  Test  Engineering  Guide 

GFE 

Government  Furnished  Equipment 

GHz 

Gigahertz 

HIL 

Hardware  in-the-Loop 

hr 

Hour 

HWCI 

Hardware  Configuration  Item 
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Hz 

Hertz 

ICD 

Initial  Capabilities  Document 

IDD 

Interface  Design  Description 

IDEF 

Integrated  Definition 

IEC 

International  Electrotechnical  Committee 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

ILS 

Integrated  Logistics  Support 

IMS 

Integrated  Master  Schedule 

INCOSE 

International  Council  on  Systems  Engineering 

IOC 

Initial  operational  capability 

IPPD 

Integrated  Product  and  Process  Development 

IPT 

Integrated  Product  Team 

IRD 

Interface  Requirements  Document 

IRS 

Interface  Requirements  Specification 

ISBN 

International  Standard  Book  Number 

ISEA 

In-Service  Engineering  Agent 

ISO 

International  Standards  Organization 

IV&V 

Independent  Verification  &  Validation 

JSSG 

Joint  Services  Specification  Guides 

KPP 

Key  Performance  Parameters 

lb 

Pound 

LRIP 

Low-Rate  Initial  Production 

LSA 

Logistics  Support  Analysis 

M&S 

Modeling  and  Simulation 

MAA 

Mission  Area  Analysis 

MAIS 

Major  Automated  Information  System 

MAPP 

Master  Acquisition  Planning  Program 

MDAP 

Major  Defense  Acquisition  Program 

MHz 

Megahertz 

mi 

Mile 

MILHDBK 

Military  Handbook 

MILSATCOMM 

Military  Satellite  Communication 

MILSTD 

Military  Standard 

min 

Minute 

MIPR 

Military  Interservice  Procurement  Request 

MNS 

Mission  Need  Statement 

MOE 

Measure  of  Effectiveness 

MOP 

Measure  of  Performance;  Memorandum  of  Policy 

MOS 

Measures  of  Suitability 

MS  or  M/S 

Milestone 

MTBF 

Mean  Time  Between  Failures 

MTTR 

Mean  Time  To  Repair 

NALCOMIS 

Naval  Aviation  Logistics  Command  Management  Information  System 

NASA 

National  Aeronautics  and  Space  Administration 

NAVAIR 

Naval  Air  Systems  Command 
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NAVAIRINST 

Naval  Air  Systems  Command  Instruction 

NAVSO 

Navy  Support  Office 

NAWCWD 

Naval  Air  Warfare  Center  Weapons  Division 

NDI 

Non-Developmental  Item 

NTAB 

Naval  Technical  Assurance  Board 

OOA 

Object  Oriented  Analysis 

OPEVAL 

Operational  Evaluation 

OPNAV 

Office  of  the  Chief  of  Naval  Operations 

Ops 

Operations 

OPTEVOR 

Operation  Test  and  Evaluation  Command 

ORD 

Operational  Requirements  Document 

OSD 

Office  of  the  Secretary  of  Defense 

OT 

Operational  Test 

OT&E 

Operational  Test  &  Evaluation 

OTRR 

Operational  Test  Readiness  Review 

P3I 

Pre-Plan  Product  Improvement 

PCA 

Physical  Configuration  Audit 

PCO 

Procurement  Contracting  Officer 

PDR 

Preliminary  Design  Review 

PDRR 

Program  Definition  and  Risk  Reduction 

PDT 

Product  Development  Team 

PEO 

Program  Executive  Officer 

PERT 

Program  Evaluation  Readiness  Technique 

PM 

Program  Manager 

PMB 

Performance  Measurement  Baseline 

PMBOK 

Project  Management  Body  of  Knowledge 

POA&M 

Plan  of  Actions  and  Milestones 

POG 

Program  Operating  Guide 

PRR 

Production  Readiness  Review 

QA 

Quality  Assurance 

QFD 

Quality  Functional  Deployment 

R&M 

Reliability  &  Maintainability 

R&D 

Research  and  Development 

RAS 

Requirement  Allocation  Sheet 

RDT&E 

Research,  Development,  Test  and  Evaluation 

RFI 

Request  for  Information 

RFP 

Request  for  Proposal 

S/N 

Serial  Number 

SBD 

Schematic  Block  Diagram 

SDD 

Software  Design  Description 

SDP 

Software  Development  Plan 

SE 

Systems  Engineering 

sec 

Second 

SEER-SEM 

Software  Development  Tool 
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SEI 

Software  Engineering  Institute 

SEMP 

Systems  Engineering  Management  Plan 

SEP 

Systems  Engineering  Plan,  formerly  Systems  Engineering  Management  Plan 

SEPWG 

Systems  Engineering  Process  Working  Group 

SFR 

System  Functional  Review 

SIL 

Software  in-the-Loop 

SLAM 

Queuing  methodology  software 

SLATE™ 

Requirements  Management  &  System  Architecture  Database  Software 

SME 

Subject  Matter  Expert 

SOO 

Statement  Of  Objectives 

SOP 

Standard  Operating  Procedures 

SOW 

Statement  of  Work 

SPC 

Statistical  Process  Control 

SPI 

Schedule  Performance  Index 

SPS 

Software  Product  Specification 

sq 

Square 

SRD 

System  Requirements  Document 

SRR 

System  Requirements  Review 

SRS 

Software  Requirements  Specification 

SSDD 

System/Segment  Design  Document 

sss 

System/Subsystem  Specification 

STR 

System  Technical  Requirements 

SV 

Schedule  variance 

SVD 

Software  Version  Description 

Syscom 

Systems  Command 

T&E 

Test  and  Evaluation 

TAA 

Team  Assignment  Agreement 

TBD 

To  Be  Determined 

TBR 

To  Be  Reviewed 

TDP 

Technical  Data  Package 

TEMP 

Test  and  Evaluation  Management  Plan 

TEPMG 

Test  and  Evaluation  Process  Working  Group 

TLS 

Time  Line  Sheet 

TOC 

Total  Ownership  Costs 

TPM 

Technical  Performance  Measures/Measurement 

TRS 

Test  Requirements  Sheet 

TWP 

Team  Work  Plan 

UML 

Unified  Modeling  Language 

VBA 

Visual  Basic  Programming 

VCRM 

Verification  Compliance  Requirement  Matrix 

W 

Watts 

WBS 

Work  Breakdown  Structure 

Wt 

Weight 

254 


Appendix  J  -  Naval  References 


Filename 

Reference  Info 

(Note:  cancelled  documents  begin  with  an  ‘X’  and  are  noted  in  the  title  these  are  the 
best  known  references  at  the  time  of  publishing,  but  are  ONLY  to  be  used  as 
references.) 

AIAA  OCD  Prep 

American  Institute  of  Aeronautics  and  Astronautics  (AIAA)  (1992). 

Operational  Concept  Document  (OCD)  Preparation  Guidelines. 

Blanchard  SE 

Blanchard,  Beniamin  S.  and  Fabryncky,  W.J.  (1997).  Systems  Engineering 
and  Analysis  (3-  ed.).  Upper  Saddle  River,  NJ:  Prentice  Hall 

Booch  OOA 

Booch,  Grady.  Object-  oriented  Analysis  and  Design  with  Applications  (2nd 
ed.)  (1994).  Santa  Clara,  CA:  Benjamin/Cummings 

C4ISR 

Department  of  Defense:  C4ISR  Architecture  Working  Group  (18  December 
1997).  C4ISR  Architecture  Framework  Version  2.0.  Arlington,  VA:  Author 

CMMIsm 

CMMIsm  Software  Engineering  Institute,  Carnegie  Mellon  University.  (2002) 

Capability  Maturity  Model  ®  Integration  for  Systems  Engineering,  Software 

Engineering,  Integrated  Product  and  Process  Development,  and  Supplier  Sourcing. 

Pittsburgh,  PA:  Author 

formerly  DAD  -  AKSS 

AT&L  Knowledge  Sharing  System  (AKSS). 

http  ://deskbook.dau.mil/j  sp/default.  j  sp 

DAU  SE  Fundamentals 

Defense  Acquisition  University  Press  (2001).  Systems  Engineering 
Fundamentals.  Fort  Belvoir,  VA:  Author 

DAU  Program  Manager’s 

Tool  Kit 

Defense  Acquisition  University  Press  (2004).  DAU  Program  Manager’s  Tool 
Kit.  Fort  Belvoir,  VA:  Author. 

DAU  Risk  Management 

Guide 

Defense  Acquisition  University  Press  (June  2003).  Risk  Management  Guide 
for  DoD  Acquisition.  Fort  Belvoir,  VA:  Author 

DSMC  Virtual 
Prototyping-Concept  to 
Production 

Defense  System  Management  College  (1993).  Press  Report  of  the  1992-1993 
Military  Research  Fellows,  Virtual  Prototyping — Concept  to  Production.  Fort 

Belvoir,  VA:  Garcia,  A.  B.,  Gocke,  R.  P.,  &  Johnson  Jr.,  N.  P. 

DD  1423-2 

Contract  Data  Requirements  List  (1996),  DD  Form  1423-2 

DD  250 

Material  Inspection  and  Receiving  Report  (2000),  DD  Form  250 

DD  254 

Contract  Security  Classification  Specifiction  (1999),  DD  Form  254 

DI-GDRO-81222 

Department  of  Defense.  Requirement  Allocation  Sheets  (RAS)  Data  Item 
Description  (DI-IPSC-81222).  Arlington,  VA:  Author 

DI-CMAN-80858B 

Department  of  Defense.  Contractor’s  Configuration  Management  Plan  (DI- 
CMAN-80858).  Arlington,  VA:  Author 

DI-ILSS-81 163  A 

Department  of  Defense.  Failure  Modes,  Effects,  and  Criticality  Analysis 
(FMEC A)  Report  (DI-ILSS-81 163 A).  Arlington,  VA:  Author 

DI-IPSC-81430 

Department  of  Defense.  Operational  Concept  Description  (OCD)  Data  Item 
Description  (DI-IPSC-81430).  Arlington,  VA:  Author 

DI-IPSC-81431 

Department  of  Defense.  System/Subsystem  Specification  (SSS)  Data  Item 
Description  (DI-IPSC-81431).  Arlington,  VA:  Author 

DI-IPSC-81432 

Department  of  Defense.  System  Architecture  Design  (SSDD)  Data  Item 
Description  (DI-IPSC-81432).  Arlington,  VA:  Author 

DI-IPSC-81433 

Department  of  Defense.  Software  Requirements  Specification  (SRS)  Data 

Item  Description  (DI-IPSC-81433).  Arlington,  VA:  Author 

DI-IPSC-81434 

Department  of  Defense.  Interface  Requirements  Specification  (IRS)  Data  Item 
Description  (DI-IPSC-81434).  Arlington,  VA:  Author 

DI-IPSC-81435 

Department  of  Defense.  Software  Design  Description  (SDD)  Data  Item 
Description  (DI-IPSC-81435).  Arlington,  VA:  Author 

DI-IPSC-81436 

Department  of  Defense.  Interface  Design  Description  (IDD)  Data  Item 
Description  (DI-IPSC-81436).  Arlington,  VA:  Author 

255 


Filename 

Reference  Info 

(Note:  cancelled  documents  begin  with  an  ‘X’  and  are  noted  in  the  title  these  are  the 
best  known  references  at  the  time  of  publishing,  but  are  ONLY  to  be  used  as 
references.) 

DI-IPSC-81437 

Department  of  Defense.  Database  Design  Description  (DBDD)  Data  Item 
Description  (DI-IPSC-81437).  Arlington,  VA:  Author 

DI-IPSC-81441 

Department  of  Defense.  Software  Product  Specification  (SPS)  Data  Item 
Description  (DI-IPSC-81441).  Arlington,  VA:  Author 

DI-IPSC-81442 

Department  of  Defense.  User  Software  Version  Description  (SVD)  Data  Item 
Description  (DI-IPSC-81442).  Arlington,  VA:  Author 

DI-MGMT-81024 

Department  of  Defense.  System  Engineering  Management  Plan  (SEMP)  (DI- 
MGMT- 8 1024).  Arlington,  VA:  Author 

DI-MISC-81180 

Department  of  Defense.  Manufacturing  Technology  Demonstration  Plan  (DI- 
MISC-81 180).  Arlington,  VA:  Author 

DI-MISC-81183A 

Department  of  Defense.  Integrated  Master  Schedule  (IMS)  Data  Item 
Description  (DI-MISC-81 183 A).  Arlington,  VA:  Author 

DI-NDTI-80566 

Department  of  Defense.  Test  Plan  (DI-NDTI-80566).  Arlington,  VA:  Author 

DoD  4245.7-M 

Department  of  Defense.  (1985)  Transition  from  Development  to  Production 
(DoD  4245.7-M).  Arlington,  VA:  Author. 

DoD  5000  Series 

Navy  Acquisition  and  Business  Management  website:  http://www.acq.osd.mil 

DoD  5000.1 

Department  of  Defense.  (2003)  The  Defense  Acquisition  System  (DoD 

5000.1).  Fort  Belvoir,  VA:  .Author.  Note:  Select  DoD  5000  Series  to  access  most 
current  issuance. 

DoD  5000.2 

Department  of  Defense.  (2003)  Operation  of  the  Defense  Acquisition  System 
(DoDI  5000.2).  Fort  Belvoir,  VA:  .Author.  Note:  Select  DoD  5000  Series  to  access 
most  current  issuance. 

DoD  5000.2-R 

Department  of  Defense.  (2003)  (Interim)  Mandatory  Procedures  for  Major 
Defense  Acquisition  Programs  (MDAPs)  and  Major  Automated  Information  System 

(MAIS)  Acquisition  Programs  (DoD  5000.2-R).  Fort  Belvoir,  VA:  .Author.  Note: 
Select  DoD  5000  Series  to  access  most  current  issuance.  To  be  replaced  with  DoD 
Acquisition  Guidebook. 

FAR/DFAR 

Federal  Acquisition  Regulation  (http://www.arnet.20v/far/)  /  Defense 

Federal  Acquisition  Regulation  (http://www.acq.osd.mil/dp/dars/dfars.html ) 

EIA-632 

Electronic  Industries  Alliance.  (1999).  EIA-632  Processes  for  Engineering  a 
System.  Arlington,  VA. 

EIA-748 

Electronic  Industries  Alliance.  (1998).  EIA-632  Earned  Value  Management 
Systems.  Arlington,  VA 

Grady  SE  Plan 

Grady,  Jeffery  O.  (1994).  System  Engineering  Planning  and  Enterprise 

Identity.  CRC  Press. 

IEEE  1220 

Institute  of  Electrical  and  Electronics  Engineers,  Information  Handling 

Services  (1999).  IEEE  Standard  for  Application  and  Management  of  the  Systems 
Engineering  Process  (IEEE  Std  1220-1998).  New  York,  NY. 

INCOSE  SE  Hdbk 

International  Council  on  Systems  Engineering  (1998)  Systems  Engineering 
Handbook  (Release  1.0).  Seattle,  W A.  www.inc0se.0r2 

IPPD  Hdbk 

Department  of  Defense  (1998).  DoD  Integrated  Product  and  Process 
Development  (IPPD)  Handbook.  Washington,  DC:  Author 

DoD  Joint  Technical 

Architecture 

DoD  JTA  as  promulgated  by  DoD  Memorandum,  dated  19  August  2003 

Lacy  SE  Mgmt 

Lacy,  James  A.  (1992).  Systems  Engineering  Management.  Jim  Lacy. 

Lake  1999 

Lake,  Jerry.  1999.  World  Class  Example.  Unpublished 

Top  11  Risk 

Office  of  Assistant  Sectretary  to  the  Navy  (RD&A)  Acquisition  and  Business 
Management.  (1998)  Top  Eleven  Ways  to  Manage  Risk.  Philadelphia,  PA: 

DODSSP 

256 


Filename 

Reference  Info 

(Note:  cancelled  documents  begin  with  an  ‘X’  and  are  noted  in  the  title  these  are  the 
best  known  references  at  the  time  of  publishing,  but  are  ONLY  to  be  used  as 
references.) 

Martin  SE  Guide 

Martin,  James  N.  (1996)  Systems  Engineering  Guidebook.  CRC  Press 

MIL-HDBK-5 1 2  A 

Department  of  Defense.  (2001)  Handbook  for  Parts  Management  (MIL- 
HDBK-512A).  Fort  Belvoir,  VA.  Author 

MIL-HDBK-6 1 A 

Department  of  Defense.  (2001)  Handbook  for  Configuration  Management 
Guidance  (MIL-HDBK-6 1  A).  Fort  Belvoir,  VA.  Author 

MIL-HDBK-245D 

Department  of  Defense.  (1996)  Handbook  for  Preparation  of  Statement  of 

Work  (SOW)  (MIL-HDBK-245D).  Fort  Belvoir,  VA.  Author 

MIL-HDBK-88 1 

Department  of  Defense.  (1998)  Work  Breakdown  Structure  (WBS)  (MIL- 
HDBK-881).  Fort  Belvoir,  VA.  Author 

MIL-STD-100G 

Department  of  Defense.  (1997)  Standard  Practice  for  Engineering  Drawings 
(MIL-STD  100G).  Fort  Belvoir,  VA:  Author 

MIL-STD-1629A 
MIL-STD-1629A  N1 

MIL-STD-1629A  N2 

MIL-STD-1629A  N3 

Department  of  Defense.  Procedures  for  Performing  a  Failure  Mode,  Effects 
and  Criticalitv  Analvsis  (MIL-STD- 1629A)  (Notes  1-3).  Arlington,  VA:  Author 

MIL-STD-498 

Department  of  Defense.  (1994)  Software  Development  and  Documentation 
(MIL-STD-498).  Fort  Belvoir,  VA:  Author 

MIL-STD-499B 

Department  of  Defense.  (XxXx)  DRAFT  Svstems  Engineering  (MIL-STD 
499B).  Fort  Belvoir,  VA:  Author 

MIL-STD-961D 
MIL-STD-961D  N1 

Department  of  Defense.  (1995)  Standard  Practice  for  Defense  Specifications 
(MIL-STD  96 ID).  Fort  Belvoir,  VA:  Author 

NAVAIR  Acquisition 

Guide 

Naval  Air  Svstems  Command  (NAVAIR).  (January  2004)  NAVAIR 
Acquisition  Guide  ( NAVAIR  #).  Author. 

NAVAIR  Design  Review 

Handbook  (AIR  4.1) 

Naval  Air  Svstems  Command  (NAVAIR).  (January  2001)  NAVAIR  Design 
Review  Handbook  (AIR  4.1)  ( NAVAIR  #).  Author. 

NAVAIR  Risk 
Management  Guide 

Naval  Air  Systems  Command  (NAVAIR).  ( expect  July  2004  approval) 

DRAFT  NAVAIR  Risk  Management  Guide  (NAVAIR  #).  Author. 

NAVAIRINST  3960.2C 

Naval  Air  Svstems  Command  (NAVAIR).  (1994)  Test  and  Evaluation 
(NAVAIRINST  3960.2C).  Author. 

NAVAIRINST  4130.1C 

Naval  Air  Svstems  Command  (NAVAIR).  (1992)  Configuration  Management 
Policv.  (NAVAIRINST  4 130. 1C).  Author. 

NAVAIRINST  4200.36C 

Naval  Air  Svstems  Command  (NAVAIR).  (2004)  Acquisition  Plans. 
(NAVAIRINST  4200.36C).  Author. 

NAVAIR  INST  4355.19B 

Naval  Air  Svstems  Command  (NAVAIR).  (2003)  Svstems  Engineering 
Technical  Review  Process.  (NAVAIR  INST  4355. 19B).  Author. 

NAVAIRINST  5000.21 

Naval  Air  Svstems  Command  (NAVAIR).  (2003)  Program/Proiect  Risk 
Management.  (NAVAIRINST  5000.21).  Author. 

NAVAIRINST  5400.154 

Naval  Air  Svstems  Command  (NAVAIR).  (2000)  Team  Assignment 
Agreement.  (NAVAIRINST  5400.154).  Author. 

NAVSO  P-6071 

Department  of  the  Navv.  (1986)  Best  Practices  (How  to  Avoid  Suprises  in  the 
World’s  Most  Complicated  Technical  Process)  The  Transition  from  Development  to 

Production.  (NAVSO  P-6071).  Author. 

Oliver  Eng  Sys 

Oliver.  David  W.,  et  al,  (1997)  Engineering  Complex  Svstems  with  Models 
and  Objects.  McGraw-Hill  Companies 

SD-2 

The  Office  of  Under  Secretary  of  Defense  for  Acquisition  and  Technology. 
(1996)  Buving  Commercial  and  Non-Developmental  Items:  A  Handbook  (SD-2). 
Philadelphia,  PA:  DODSSP 

SD-5 

The  Office  of  Under  Secretary  of  Defense  for  Acquisition  and  Technology. 
(1997)  Market  Research  (SD-5).  Philadelphia,  PA:  DODSSP 

Yourdon 

Yourdon,  Edward.  (1989).  Modem  Stmctured  Analvsis.  Upper  Saddle  River, 
NJ.  Prentice  Hall 

257 


258 


AcqWeb  -  Welcome 


Home  I  Search 


Contact  Us 


Search 


Acquisition,  Technology,  &  Logistics 

"ACQWeb  Highlights 

J 


ECURIT  Y  PRIVACY  DISCLAIMER 


Submit 


Welcome 


Home 

AT&L  Documents 

Acquisition  FAQs 
Archived  Highlights 

□ 


Navigate 


Office  Navigator 

DoD  Root  Certificate 

Download 

Search 

Help 

Feedback 

□ 


Top  Sites 


Video  Services 

DFARS 

DoD  5000.1  &  5000.2 

AKSS/Deskbook 

UAV  Roadmap 
Contract  Pricing  Guides 

□ 


Related  Sites 

I 


Defense  LINK  -  The  U.S. 


X 


FIRSTGOV 

a - 


The  U.S. 


July  27,  2004 


Contact  the  Webmaster  at: 
ATL- 

Webmaster@osd.mil 


I&E  Business 
Transformation  Seal 


The  I&E  Business  Transformation  Directorate  was 


established  in  May  of  2003  to  lead  process  change  across 
all  installation  and  environment  business  areas  and  to  support  I&E  Domian 
governance  within  the  Business  Management  Modernization  Program 
(BMMP). 


Department  of 
Defense  Military 


Department  of  Defense  Military  Equipment  Valuation 


The  PP&E  Policy  Office,  OUSD(AT&L),  leads  the  Department’s  efforts  to 
implement  the  new  Federal  Accounting  Standards  Advisory  Board’s  (FASAB) 
accounting  and  reporting  standard  for  military  equipment  in  order  to  ensure  that 
a  standard,  consistent  approach  is  used  across  the  Department.  For  more 
information,  please  visit  the  Military  Equipment  Valuation  website. 


Corrosion  Prevention  and  Control  Program  Training 

Corrosion  Prevention  and  Control  Program  training  is  now  available  at  CPC 
Training.  It  is  designed  to  assist  in  the  implementation  of  the  policy  directed  by 
the  Acting  USD  (AT&L)  Memorandum  of  12  November  2003  that  states  that 
acquisition  programs  are  to  address  corrosion  prevention  at  the  earliest  stages  of 
development  and  that  Corrosion  Prevention  and  Control  (CPC)  planning  is  to  be 
addressed  in  conjunction  with  milestone  reviews.  CPC  Program  training  CDs 
can  be  requested  at  cpcprogramtraining  @  dau.mil 


Energy  2004  Workshop 


The  Energy  2004  workshop,  scheduled  for  August  8-11  in  Rochester,  NY,  is 
designed  for  federal,  state,  local  and  private  sector  energy  managers,  energy 
service  companies,  utilities,  procurement  officials,  engineers  and  other  energy 
professionals.  Topics  that  will  be  covered  include  establishing  or  improving  an 
energy  management  program,  procuring  renewable  and  energy-efficient 
products  and  services,  and  incorporating  sustainable  design  concepts.  For  more 
information,  please  visit  the  Energy  2004  website. 


The  Acquisition  Community  Connection  (ACC)  is  the  collaborative  arm  of 
the  AT&L  Knowledge  System  that  complements  the  existing  information 
resources  located  on  the  AT&L  Knowledge  Sharing  System  (AKSS).  The 
ACC  consists  of  publicly  accessible  knowledge  communities  whose  goal  is 
connecting  people  with  know-how  across  all  DoD  organizations  and  industry. 


□ 


□ 


□ 


ACQWeb  is  the 
online  home  of  the 
Office  of  the  Under 
Secretary  of  Defense 
for  Acquisition, 
Technology,  and 
Logistics. 

To  learn  more  about 
ACQWeb  or  find 
answers  to  your 
technical  problems, 
please  visit  our  help 
section. 


□ 

Thank  you  for  using 
ACQWeb!  Please 
contact  the  webmaster 
with  any  questions, 
comments  or 
suggestions.  We  are 
always  looking  for 
ways  to  help  you  get 
the  information  you 
need. 


Featured  Site  of  the 

Month 


Link  for  Advanced 
Systems  &  Concepts 


Q 


http://www.acq.osd.mil/  (1  of  2)  [7/27/2004  1 1 :57:06] 


AcqWeb  -  Welcome 


This  page  was  last 
updated: 

Thursday,  7/1/04  8:46 


The  Deputy  Under 
Secretary  of  Defense 
(Advanced  Systems  & 
Concepts)  is 
responsible  to  the 
USD  (AT&L)  for 
oversight  technical 
transition  from 
Science  & 

Technology  to 
Defense  acquisition 
efforts,  and  oversight 
of  joint 

experimentation 
supporting  this 
technology  transition. 

□ 


http://www.acq.osd.mil/  (2  of  2)  [7/27/2004  1 1 :57:06] 


AT&L  Knowledge  Sharing  System 


jfT&L  Knowledge  She 

four  one  stop  source  for  AT&L  information 

Home  I  DAU I  Contact  /  FAQ  I  Site  Map  /  Help  /  Advanced  Search 


AKSS  Today!  What's  new  in  DoD  acquisition. 
AT&L  Knowledge  Sharing  Update 


New  JCIDS/DoD  5000.1'PPBE 
Policy  is  Available 

click  here 


We  want  your,  * 

' 

feedback-  {{{§ 

again! 

_ / 

DAU  Deploys  AT&L  Knowledge  Sharing  System  (AKSS)  3.0  with  Interwoven: 

DAU  has  procured  and  transitioned  the  AKSS  to  the  Interwoven  Content 
Management  System  (CMS),  in  a  technical  upgrade  designed  to  improve  the 
timeline  for  adding  and  correcting  the  content  of  AKSS.  To  the  user,  this  means 
that  new  links  and  corrections  to  golden  sources,  Acquisition,  Technology,  and 
Logistics  (AT&L)  web-sites,  training  information,  guidebooks  and  handbooks,  and 
other  menu  driven  content,  can  be  added  to  AKSS  almost  instantly.  Additionally, 
hot  topics  and  suggested  AT&L  news  articles  can  be  posted  to  AKSS  on  the  same 
day  that  they  appear  on  the  web.  The  user  will  not  see  any  change  in  the 
appearance  or  functionality  of  the  AKSS,  but  will  find  that  broken  or  mis-identified 
links  will  be  fixed  or  updated  within  minutes  of  discovery.  This  new  CMS 
capability  ensures  that  AKSS  3.0  will  remain  a  top  resource  for  mandatory  AT&L 
policy  and  information. 

(entire  article) 

Pilot  System  to  Analyze  Defense  Spending  : 

The  Defense  Department  has  launched  a  pilot  to  help  military  brass  understand 
what  they’re  buying  and  where  they’re  buying  it.  "The  department  is  building  a 
system  to  pull  data  from  disparate  databases  for  analysis  by  DOD  buying  teams," 
said  Mark  Krzysko,  deputy  director  of  Defense  procurement  and  acquisition 
policy.  The  department  expects  to  begin  testing  the  first  iteration  of  the  11 -month, 
$950,000  pilot  in  October.  The  project  is  funded  by  the  Navy  Department’s 
eBusiness  Operations  Office,  which  recently  identified  four  projects  to  begin  pilot 
implementation  under  the  Rapid  Acquisition  Incentive-Net  Centricity  program  for 
2004.  Krzysko  said  users  will  be  able  to  identify  procurement  trends,  buying 
patterns  and  opportunities  for  strategic  purchasing,  which  will  result  in  cost 
savings  and  quality  improvements.  A  negative  report  last  year  from  the  General 
Accounting  Office  was  one  catalyst  for  the  project,  officials  said.  Auditors  reported 
that  DoD  needed  to  improve  the  way  it  manages  the  $100  billion  it  spends  on 
services  contracts. 

(entire  article) 

Naval  Transformation  Roadmap: 


•  CJCS  Instructions  31 70.01  D 

.  CJCS  Manual  3170.01 

.  DoDD  5000.1 

.  DoDI  5000.2 

•  FMS  Manual 

•  DCMA  Instruction/Guidebook 

•_DCAA  Manual 

t  FAR 
.  DFARS 

•  Other  FAR  Sudds 

.  DoD  7000.1 4-R 

•EI  Toolkit 


Suggested  Reading 


•  Joint  Tactical  Radio  System  ( JTRS )  Architecture  is 

the  Basis  for  International  Commercial  Standards 

• _Combatant  Commanders  Need  Adaptive,  Agile 

Forces 

•  Army  announces  modularization  schedule  through 

FY07 

•  U.S.  Army  recognizes  its  Top  Ten  Greatest 

Inventions 


Suggested  Reading  Previous  Postings 


Naval  transformation  will  support  joint  transformation  by  delivering  new  military 
capabilities  and  dramatically  enhancing  current  capabilities  to  protect  and 
advance  America’s  worldwide  interests  by  assuring  access  and  projecting  power 
from  the  sea.  While  the  Navy  -  Marine  Corps  Team  is  expanding  the  entire  array  of 

http://akss.dau.mil/jsp/default.jsp  (1  of  2)  [7/27/2004  12:05:42] 


niiiiiinini 

New  Policy  Documents 

HI 

1.  1  \  1 

AT&L  Knowledge  Sharing  System 


AKSS-CD 


Version 

September  2003 
is  now  available 


dick  here  to 
reserve  your  copy 

* 


naval  capabilities  we  provide  the  Nation,  our  transformation  is  centered  upon  the 
development  of  Seabasing:  the  concepts  and  capabilities  that  exploit  our 
command  of  the  sea  to  project,  protect,  and  sustain  integrated  warfighting 
capabilities  from  the  maritime  domain.  Seabasing  and  the  supporting  tools  we  are 
developing  will  usher  in  dramatic  new  ways  of  employing  naval  forces  to  deter 
conflict  and,  when  required,  to  wage  war.  Throughout,  every  aspect  of  naval 
transformation  will  be,  first  and  foremost,  committed  to  and  built  upon  the 
principles  of  jointness.  Seabasing  will  provide  new  naval  capability  options  for  use 
by  Joint  Force  Commanders  in  innovative  combinations  with  the  transformed 
capabilities  of  the  other  Services  and  Agencies. 

(entire  article) 


• _OUSD  Memo  Instructions  for  Modular  Open 

Systems  Approach  ( MOSA )  Implementation 

•  OUSD  Memo  Amplifying  DoDD  5000. 1  Guidance 

Regarding  Modular  Open  Systems  Approach 

(MOSA)  Implentation 

•  General  Services  Administration  Acguisition  Manual 

•  Navy  Marine  Corps  Acguisition  Guide  ( Revised ) 

•  CJCSI  62 12. QIC  INTEROPERABILITY  AND 

SUPPORTABILITY  OF  INFORMATION 
TECHNOLOGY  AND  NATIONAL  SECURITY 

SYSTEMS 


-»  AKSSTodav  Previous  Postings 


What  Do  You  Think 

Have  you  joined  a  community  of  practice  (COP)  ? 


o 

No 


O  What's  a  COP? 


Tip  of  the  Week 


\  Found  a  Dead  Link? 


News 


•  DoD 

•  Army 

•  Navy 


Web  Help  Desk 

Privacy  and  Security  /  Document  Management  \  Contact  Us  /  Feedback  /  Legal  Notices 

ISSC@dau.mil 

http://akss.dau.mil/jsp/default.jsp  (2  of  2)  [7/27/2004  12:05:42] 


INTERNATIONAL  COUNCIL  ON  SYSTEMS  ENGINEERING  (INCOSE) 


Join  INCOSE  Renew 


intetn&fmna}  Council  on  Sy^torm  Engine  onng 


The  International  Council  on  Systems  Engineering  is  a  not-for-profit 
membership  organization  founded  in  1990.  INCOSE  is  an  international 
authoritative  body  promoting  the  application  of  an  interdisciplinary 
approach  and  means  to  enable  the  realization  of  successful  systems. 


Acprancing  the  Practice 

INCOSE 
200  i  j| 

■  sIIl.SMlTI  .  Sly 

BjprpppB 

1  : 

G2  S  EDoK 

i  a  ii _ a  :  —  n.  _ 

Engineering 
Guide  to  SE  Body 
of  Knowledge 
Strategic  Initiatives 
Technical 
Activities 

Standards  Update 

Research 

Partnerships 

l 


Quick  Links 


Find  am  SE  Tool 
Academic  Program  Directory 
Job  Bank 
My  Chapter 


Current  News 


J 

— - - i 

Media 

!p 

Relations 

h 


New  INCOSE  Website  (05  Jul  04) 

Though  we  are  still  wrapping  up  the  content 
and  fixing  a  few  issues,  we  hope  you  enjoy 
the  new  INCOSE  website. 

INCOSE  Products  Area  Expanded  (05  Jun 

04) 

INCOSE  has  classified  and  published  all 
existing  INCOSE  Systems  Engineering 
Products. 

Call  for  Papers  (05  Jun  04) 

Share  your  knowledge  and  experience  at  a 
future  INCOSE  event. 

Defense  Acquisition  University  Joins  the 
INCOSE  CAB  (01  Jun  04) 

We  welcome  Defense  Acquisition  University 
(DAU)  as  the  newest  member  of  our 
Corporate  Advisory  Board. 

Systems  Engineering  Handbook  v2a 
Released  (01  Jun  04) 

Version  2a  of  the  INCOSE  Systems 
Engineering  Handbook  is  now  available. 


View  All  News 


Upcoming 
Events 


September  15-18, 
2004 

ICSE  and  INCOSE 
2004  Region  II 
Conference 


October  25  -  28,  2004 

National  Defense 
Industrial  Association 
(NDIA)  7th  Annual 
Systems  Engineering 
Conference 


November  02  -  4, 
2004 

INCOSE  Mid-Atlantic 
Regional  Conference 
2004  (MARC2004) 


November  08-10, 
2004 

Systems 

Engineering/Test  and 
Evaluation  Conference 
(SETE)  2004 


http://www.incose.org/  (1  of  2)  [7/27/2004  12:10:25] 


INTERNATIONAL  COUNCIL  ON  SYSTEMS  ENGINEERING  (INCOSE) 


January  28  - 
February  03,  2005 

INCOSE  International 
Workshop  2005 


View  All  Events 


©  Copyright  1996-2004  International  Council  on  Systems  Engineering.  All  Rights  Reserved. 
Contact  INCOSE:  +1  206-361-6607  or  800-366-1 164  (toll-free  U.S.),  info@incose.org 


Terms  of  Use  |  Privacy  Statement 


http://www.incose.org/  (2  of  2)  [7/27/2004  12:10:25] 


Defense  Acquisition  Regulations  Directorate 


Organization 


PAR  Directorate 

PAR  Council 


Defense  Acquisition 

Regulations  Directorate 

What's  New?.  Subscribe  to  DFARS  News  DFARS  Contact 


Defense  FAR  Supplement  (DFARS) 


Please  visit  http://www.acq.osd.mil/dpap/dfars/index.htm  for  the  latest  version  of  the  DFARS 

(DFARS)  from  the  Director  of  Defense  Procurement. 


PAR  Committee 

DFARS  Transformation 

Homepage 


FAR 


GSA 

Hill  AFB 


DFARS 


DFARS 

Change  Notices 


The  1998  Edition  of  the  DFARS  replaces  the  1991  Edition.  See  DFARS 
Change  Notices  for  links  to  descriptions  of  all  changes  to  the  1998  edition. 

Browse  the  DFARS  text  in  HTML  format.  (Best  viewed  in  Internet  Explorer 
6.0) 

Browse  the  DFARS  text  in  PDF. 

Browse  the  DFARS  text  in  MS  Word  7.  (To  view  in  the  correct  format  you 
must  have  the  Century  Schoolbook  font.)  Download  Century  schoolbook  font. 


Hill  AFB 


Order  Code  Assignments 

(formerly  in  Appendix  G) 


Subscribe  to 
News  Alert 


FAR  News 
DFARS  News 


Class  Deviations 


Class  Deviations 


Guides 


AT&L  Knowledge  Sharing 

System 
Contract  Pricing 

Small  Entity  Compliance 

Operating  Guide 


Out  for  Public  Comment 


FAR  Rules 
DFARS  Rules 


Download  Zip  files  of  the  DFARS  in  PDF. 

Download  Zip  files  of  the  DFARS  in  MS  Word. 

Search  either  the  current  DFARS  or  previous  versions  of  the  DFARS  (1998 
Edition  only). 

To  receive  the  latest  news  of  changes  to  the  DFARS,  subscribe  to  the  DFARS 
News.  The  DFARS  News  will  keep  you  informed  by  electronic  mail  about 
changes  to  the  DFARS,  proposed  rules,  public  meetings,  and  other  DFARS- 
related  issues. 

To  recommend  editorial  changes  to  the  DFARS,  such  as  corrections  of 
misspelled  words,  omitted  words  or  lines,  or  errors  in  format,  submit  an 
electronic  message  to  dfars@osd.mil .  The  message  should  include  the 
DFARS  citation  and  a  complete  description  of  the  error.  To  recommend 
substantive  changes  to  the  DFARS,  follow  the  instructions  in  DFARS  201 .201  - 
1(d). 


Info.  Collection 


Requirements 


View  DFARS  Public 
Comments 


http://www.acq.osd.mil/dp/dars/dfars.html  (1  of  2)  [7/27/2004  12:10:41] 


Defense  Acquisition  Regulations  Directorate 


DFARS  Pub.  Comments 


Status  of  Open  Cases 


FAR 

DFARS 

Mgt.  Status  Report 


FAQs 


FAQs 


Security  &  Privacy 
Disclaimer 


Security  &  Privac  Disclaimer 


Back  to  the  Defense 
Procurement  Home  Page,  or 
go  to  ACQWeb,  from  the  Under 


Secretary  of  Defense  for 
Acquisition,  Technology  and 
Logistics. 


fax  (703)  602  0350  Site  modified  daily 


http://www.acq.osd.mil/dp/dars/dfars.html  (2  of  2)  [7/27/2004  12:10:41] 


EMAIL  dfars@osd.mil 


Defense  Procurement  and  Acquisition  Policy  -  Redirect 


The  content  that  you  have  tried  to  access  was  on  a  Web  Site  that  has 
now  been  merged  in  to  the  DPAP  Web  Site. 

You  will  be  redirected  to  the  DPAP  homepage  in  a  few  seconds. 

You  may  also  click  ahead,  http://www.acg.osd.mil/dpap/ 


http://www.acq.osd.mil/dpap/general/redirect.htm  [7/27/2004  1 2:1 0:55] 


Defense  Procurement  and  Acquisition  Policy 

Acquisition  Now  I  Acquisition  Today  I  Knowledge  Management  I  AKSS  (Deskbook)  I  PAR  Council  I  E-Business 


DPAP  Seal 

Rnttnm _ 


Defense  Procurement  and  Acquisition  Policy 

.1 - 


Organization 
Policies,  Regulations,  &  Laws 

Workforce,  Training  &  Career 

Development 

Industry  Forums 

Special  Projects 

Links 


IRAQ 

Donate  to  US 

Reconstruction 

Troops  and 

http://www.acq.osd.mil/dpap/  [7/27/2004  12:1 1 :44] 


Federal  Acquisition  Regulation  (FAR)  Home  Page 


FAR  Reissue  2001  -PDF  VOL  I  1  FAR  Reissue  2001  -  PDF  VOL  II 
(For  Printing  Purposes  Only)  I  (For  Printing  Purposes  Only) 


All  PDF  Files  Require  Adobe  Acrobat  5.0 


Proposed  Rules 
Public  Comments 
FACs  -  Looseleaf 
FACs  -  Federal  Register 
Small  Entity  Compliance  Guides 


FAR  Reference 


FAR  (Archived)  -  HTML 
FAR  (Archived) -PDF 
GSA  Forms  Library 
FAR  -  Zipped 
Search  the  FAR 


Other  Information 


Subscribe  to  FAR  News 
Frequently  Asked  Questions 
Authority  of  the  FAR 
FAR  Drafting  Guide 
E-Handouts 


For  Persons  with  Disabilities 


http://www.arnet.gov/far/  [7/27/2004  12:12:16] 


Software  Engineering  Institute  (SEI)  Home  Page 


Saorch  Contact  Us  Site  Map  What's  Now 


Carnegie  Mr  Niki 

Software  En  g  i  nearing  Institute  ^ - 

Quick  Scorch 


ABOUT  THE  SEI 

Our  Sponsor 
Carnegie  Mellon 
SEI-Europe 
Permissions 


SEI  AREAS  OF  WORK 

Management 

Engineering 

Acquisition 

WORK  WITH  US 

Contracting 

Employment  Opportunities 
Licensing 

PRODUCTS  AND  SERVICES 

Conferences  &  Events 
Education  &  Training 
Licensed  Providers 
Membership  Program 


PUBLICATIONS 

Addison-Wesley  SEI  Series 
Annual  Report 
news@sei 
Technical  Reports 


For  more  information  about 
the  SEI,  its  products,  services, 
courses,  and  events,  contact 

Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213-3890 
412-268-5800 
http://www.sei.cmu.edu 


SEI 


TCS  America  Joins  International  Process  Research  Consortium 

Carnegie  Mellon  University  Names  Paul  D.  Nielsen 

Director  of  the  Software  Engineering  Institute 

Register  for  Third  Software  Product  Line  Conference  (SPLC  2004), 

to  Be  Held  in  Boston,  August  30 — September  2 

First  Three  Keynote  Speakers  Announced  for  SEPG  2005, 

the  Premier  Process  Improvement  Conference 

Software  Engineering  Information  Repository  (SEIR) 

Membership  Soars 

SEI  Public  and  Media  Relations 


The  right  software,  delivered  defect  free,  art  time  and  on  cost,  every  time. 


Select  a  topic 


http://www.sei.cmu.edu/  (1  of  2)  [7/27/2004  13:54:01] 


Software  Engineering  Institute  (SEI)  Home  Page 


Search  I  Contact  Us  I  Site  Map  I  What's  New  I  About  the  SEI  I  SEI  Areas  of  Work 
Work  with  Us  I  Products  and  Services  I  Publications  I  Terms  of  Use 


Copyright  2004  by  Carnegie  Mellon  University  Terms  of  Use 


http://www.sei.cmu.edu/  (2  of  2)  [7/27/2004  13:54:01] 


y  /  1  y  J  I  \ 
A\  r  n  fl  (  C1  I 


U.S,  D&D/DS/8E 


Policies  and 
Procedures 


Related 

Websites 


Papers  and 
Speeches 


Systems  Engineering 
Homepage 


I 


Events  I  Tools  and  Products 


Description: 


J 


Related 

Publications 

I 


J 


Risk  Management  is  a  systematic  approach  to  identifying, 
analyzing,  and  controlling  areas  or  events  with  a  potential  for 
causing  unwanted  change.  It  is  through  risk  management  that  risks 
to  the  program  are  assessed  and  systematically  managed  to  reduce 
risk  to  an  acceptable  level. 

Definitions: 

Risk  is  a  measure  of  the  inability  to  achieve  overall  program 
objectives  within  defined  cost,  schedule,  and  technical  constraints 
and  has  two  components:  (1)  the  probability  of  failing  to  achieve  a 
particular  outcome  and  (2)  the  consequences  of  failing  to  achieve 
that  outcome. 

Risk  Management  is  the  act  or  practice  of  controlling  risk.  It 
includes  risk  planning,  assessing  risk  areas,  developing  risk¬ 
handling  options,  monitoring  risks  to  determine  how  risks  have 
changed,  and  documenting  the  overall  risk  management  program. 


Risk  has  always  been  a  concern  in  the  acquisition  of  DoD  systems. 
The  acquisition  process  itself  is  designed,  to  a  large  degree,  to  allow 
managers  to  control  events,  or  their  consequences,  that  might 
adversely  affect  a  program.  In  the  past,  many  managers  viewed  risk 
as  something  to  be  avoided  and  required  that  any  program  that  had 
risk  areas  be  subjected  to  review  and  oversight.  This  attitude  has 
changed.  DoD  decision  makers  recognize  that  risk  is  inherent  in 
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programs,  and  a  goal  of  DoD  acquisition  is  to  study  future  program 
events,  identify  potential  risks,  and  take  measures  to  control  them 
and  ensure  favorable  outcomes. 

There  is  no  one  standard  approach  for  risk  management.  The 
approach  taken  must  be  tailored  to  the  specific  program  taking  into 
account  program  constraints  and  the  acquisition  strategy.  There  are 
some  common  elements  of  successful  risk  management  efforts: 

•  Recognition  that  risk  management  is  a  program  management 
responsibility 

•  The  risk  management  process  includes: 

planning  for  risk  management, 

">  continuously  identifying,  analyzing  program  events, 

">  assessing  the  likelihood  of  their  occurrence  and 
consequences, 

' >  incorporating  handling  actions  to  control  risk  events, 

>  and  monitoring  a  program's  progress  toward  meeting 
program  goals. 

The  Systems  Engineering  group,  within  Interoperability  (10) 
organization  is  responsible  for  risk  management  in  DoD  and  has,  at 
the  direction  of  the  Undersecretary  of  Defense,  Acquisition, 
Technology,  and  Logistics  (USD  (AT&L)),  examined  DoD's 
approach  to  managing  risk.  Systems  Engineering  formed  a  Working 
Group  (RMWG),  composed  of  representatives  from  the  Services 
and  other  DoD  agencies  involved  in  systems  acquisition,  to  assist  in 
the  evaluation  of  the  Department's  approach  to  risk  management. 
The  results  were  briefed  to  the  Defense  Manufacturing  Council 
(DMC),  which  directed  DTSE&E  to  incorporate  any  guidance  and 
advice  in  the  Defense  Acquisition  Deskbook  (DAD).  The  DAD, 
Section  2.5.2,  Risk  Management,  is  the  result.  This  work  also 
provided  the  basis  for  the  Risk  Management  Guide  produced  by  the 
OSD,  Defense  Acquisition  University  (DAU),  and  Defense  Systems 
Management  College  (DSMC). 
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The  Working  Group  will  continue  to  provide  a  forum  for  sharing 
experiences  and  knowledge  in  order  to  provide  Program  Managers 
with  the  latest  tools  and  advice  on  managing  risk.  You  are  invited  to 
share  your  experiences  and  seek  advice  on  the  latest  techniques. 


SE  CoP 


Systems  Engineering  Community  Of  Practice  (SE  CoP)  -  The 

SE  CoP  is  one  of  six  communities  of  practice  (and  10  special 
interest  areas)  located  in  the  "Acquisition  Community  Connection 
(ACC)"  website.  The  ACC  was  designed  for  the  purpose  of 
providing  authoritative  acquisition,  technology  and  logistics 
information  and  access  to  experts  and  peers  working  on  critical 
AT&L  processes. 
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